Introduction to Computer Vision: Plant Seedlings Classification¶
Problem Statement¶
Context¶
In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.
Objective¶
The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.
Data Dictionary¶
The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.
The dataset can be download from Olympus.
The data file names are:
- images.npy
- Labels.csv
Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.
The goal of the project is to create a classifier capable of determining a plant's species from an image.
List of Species
- Black-grass
- Charlock
- Cleavers
- Common Chickweed
- Common Wheat
- Fat Hen
- Loose Silky-bent
- Maize
- Scentless Mayweed
- Shepherds Purse
- Small-flowered Cranesbill
- Sugar beet
Note: Please use GPU runtime on Google Colab to execute the code faster.¶
Importing necessary libraries¶
# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
#!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==2.03 opencv-python==4.8.0.76 -q --user
#!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 opencv-python==4.8.0.76 -q --user
# Installing the libraries with the specified version.
# uncomment and run the following lines if Jupyter Notebook is being used
#!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.11.1 matplotlib==3.3.4 numpy==1.24.3 pandas==1.5.2 opencv-python==4.8.0.76 -q --user
#!pip install pandas==2.0.3 -q --user
#!pip install typing-extensions==4.6.0 -q --user
#!pip install scikeras==0.11.0 keras==2.15.0 scikit-learn==1.4.2 -q --user
Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.
import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import math
import cv2
import seaborn as sns
# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization
from tensorflow.keras.optimizers import Adam,SGD
from tensorflow.keras import optimizers
from sklearn import preprocessing
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix,classification_report
from sklearn.preprocessing import LabelBinarizer
#Importing ImageGrid to plot the plant sample images
from mpl_toolkits.axes_grid1 import ImageGrid
from sklearn.model_selection import train_test_split
from tensorflow.keras import backend
from keras.callbacks import ReduceLROnPlateau
import random
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')
2024-07-12 16:04:19.866070: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
#Importing cv2_imshow for displaying images, but only if running on COLAB
try:
from google.colab.patches import cv2_imshow
IN_COLAB = True
except:
IN_COLAB = False
SHARPEN = False #should we sharpen images
BLUR = False #should we blur images NOTE, we would only do one or the other, so could use 1 boolean
RESIZE = True #should we resize images
epochs = 25
batch_size = 32
print(SHARPEN)
print(RESIZE)
print(BLUR)
print(epochs)
print(batch_size)
False True False 25 32
print(IN_COLAB)
False
print(tf.__version__)
print(sns.__version__)
print(np.__version__)
print(pd.__version__)
print(cv2.__version__)
#opencv-python.__version__
print(np.__version__)
2.15.0 0.11.1 1.24.3 1.5.2 4.8.0 1.24.3
Loading the dataset¶
# Uncomment and run the below code if you are using google colab
if IN_COLAB:
print("Running in Colab")
from google.colab import drive
drive.mount('/content/drive')
# Load the image file of dataset
# Load the labels file of dataset
if IN_COLAB:
print("Running in Colab")
images = np.load("/content/drive/MyDrive/Colab Notebooks/CV/images.npy")
labels = pd.read_csv("/content/drive/MyDrive/Colab Notebooks/CV/Labels.csv")
else:
images = np.load('images.npy')
labels = pd.read_csv('labels.csv')
Data Overview¶
Understand the shape of the dataset¶
print(images.shape)
print(labels.shape)
(4750, 128, 128, 3) (4750, 1)
Creating a list of category names¶
labels_plants = np.unique(labels)
print(labels_plants)
#store this for use elsewhere
num_labels = len(labels_plants)
print(num_labels)
['Black-grass' 'Charlock' 'Cleavers' 'Common Chickweed' 'Common wheat' 'Fat Hen' 'Loose Silky-bent' 'Maize' 'Scentless Mayweed' 'Shepherds Purse' 'Small-flowered Cranesbill' 'Sugar beet'] 12
Observations:¶
- There are 12 groupings of image types
- A total of 4750 plant images
- Each image is size 128x128 pixels
- The number of channels is 3, images are RGB (Red, Green, Blue)
Exploratory Data Analysis¶
# switch to images copy so we can not have to reload if possible when we change things
images_copy = images.copy()
#Show a single sample image
if IN_COLAB:
print("Running in Colab")
cv2_imshow(images_copy[1000])
else:
plt.imshow(images_copy[1000])
Observations¶
- We can observe that the images are being shown in BGR when plotted with openCV. We will need to convert to RGB for better viewing and recognition
#Show sample images
#defining a grid of size 12X12
fig = plt.figure(1, figsize=(num_labels, num_labels))
grid = ImageGrid(fig, 111, nrows_ncols=(num_labels, num_labels), axes_pad=0.05)
i = 0
index = labels.index
# 12 images from each category
for category_id, category in enumerate(labels_plants):
condition = labels["Label"] == category
plant_indices = index[condition].tolist()
for j in range(0,num_labels):
ax = grid[i]
image_rgb = cv2.cvtColor(images_copy[plant_indices[j]], cv2.COLOR_BGR2RGB) # Adding subplots with 3 rows and 4 columns
ax.imshow(image_rgb)
ax.axis('off')
if i % num_labels == num_labels - 1:
#print the names for each category
ax.text(200, 70, category, verticalalignment='center')
i += 1
plt.show();
if IN_COLAB:
print("Running in Colab")
cv2_imshow(images_copy[1000])
else:
plt.imshow(cv2.cvtColor(images_copy[1000], cv2.COLOR_BGR2RGB))
plt.rcParams["figure.figsize"] = (12,5)
sns.countplot(x=labels.iloc[:,-1],order = labels['Label'].value_counts().index, palette='Spectral')
plt.xlabel('Plant Categories')
plt.xticks(rotation=45)
(array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]), [Text(0, 0, 'Loose Silky-bent'), Text(1, 0, 'Common Chickweed'), Text(2, 0, 'Scentless Mayweed'), Text(3, 0, 'Small-flowered Cranesbill'), Text(4, 0, 'Fat Hen'), Text(5, 0, 'Charlock'), Text(6, 0, 'Sugar beet'), Text(7, 0, 'Cleavers'), Text(8, 0, 'Black-grass'), Text(9, 0, 'Shepherds Purse'), Text(10, 0, 'Common wheat'), Text(11, 0, 'Maize')])
Observations:¶
- This data set is imbalanced
- "Loose Silky-bent" plant images are the largest group, followed by "Common Chickweed" and "Scentless Mayweed"
- The smallest image groupings are "Sheperds Purse", "Common Wheat" and "Maize"
Data Pre-Processing¶
# switch paths to have a processed files
images_to_process = images_copy.copy() #leaving in place in case I decide to look at lap
images_to_resize = images_copy.copy()
Convert and/or resize the images based on flags set earlier¶
images_to_resize.shape
(4750, 128, 128, 3)
As we can see, the images are currently 128 x 128 x 3
# Resizing the image size to half ie., from 128X128 to 64X64 and converting to RGB
images_color_resized=[]
for i in range(len(images_to_resize)):
#Apply sharpen filter
if SHARPEN:
kernel = np.array([[0, -1, 0],
[-1, 5,-1],
[0, -1, 0]])
images_to_resize[i] = cv2.filter2D(src=images_to_resize[i], ddepth=-1, kernel=kernel)
#Apply Gaussian blur
if BLUR:
images_to_resize[i] = cv2.GaussianBlur(images_to_resize[i],(5,5),0)
#Convert the BGR images to RGB images
images_to_resize[i] = cv2.cvtColor(images_to_resize[i], cv2.COLOR_BGR2RGB)
#Resize images
if RESIZE:
images_color_resized.append( cv2.resize(images_to_resize[i], None, fx=0.50, fy=0.50, interpolation=cv2.INTER_LINEAR))
else:
images_color_resized.append( cv2.resize(images_to_resize[i], None, fx=1.0, fy=1.0, interpolation=cv2.INTER_LINEAR))
np.shape(images_color_resized)
(4750, 64, 64, 3)
Observations¶
- We can see the images are now 64x64 in size.
- They retain the color channels.
Image before resizing
print('\n')
print("Image before resizing")
if IN_COLAB:
print("Running in Colab")
cv2_imshow(images_copy[1000])
else:
plt.imshow(cv2.cvtColor(images_copy[1000], cv2.COLOR_BGR2RGB))
Image before resizing
Observations¶
This image is pretty blurry already, suggesting it was resized once already. Resizing again may cause it to lose more data, affecting accuracy.
Image resized with no sharpening
image_resize = cv2.resize(images_copy[1000], None, fx=0.50, fy=0.50, interpolation=cv2.INTER_LINEAR)
print('\n')
print("Image resized with no sharpening/blurring")
if IN_COLAB:
print("Running in Colab")
cv2_imshow(images_copy[1000])
else:
plt.imshow(cv2.cvtColor(image_resize, cv2.COLOR_BGR2RGB))
Image resized with no sharpening/blurring
Observations¶
After resizing, this image becomes much less distinctive, suggesting resizing may affect accuracy.
Image after sharpening before resizing
kernel = np.array([[0, -1, 0],
[-1, 5,-1],
[0, -1, 0]])
image_sharp = cv2.filter2D(src=images_copy[1000], ddepth=-1, kernel=kernel)
print('\n')
print("Original image after sharpening")
if IN_COLAB:
print("Running in Colab")
cv2_imshow(images_copy[1000])
else:
plt.imshow(cv2.cvtColor(image_sharp, cv2.COLOR_BGR2RGB))
Original image after sharpening
Observations¶
Sharpening this image makes the edges appear a bit more distinctive and may help in identifying the plant.
image_blur = cv2.GaussianBlur(images_copy[1000],(5,5),0)
print('\n')
print("Original image after blurring")
if IN_COLAB:
print("Running in Colab")
cv2_imshow(images_copy[1000])
else:
plt.imshow(cv2.cvtColor(image_blur, cv2.COLOR_BGR2RGB))
Original image after blurring
Observations¶
Blurring this image makes the edges much softer. This may not lead to good identification accuracy.
Image after resizing
kernel = np.array([[0, -1, 0],
[-1, 5,-1],
[0, -1, 0]])
image_sharp = cv2.filter2D(src=images_copy[1000], ddepth=-1, kernel=kernel)
image_sharp_resize = cv2.resize(image_sharp, None, fx=0.50, fy=0.50, interpolation=cv2.INTER_LINEAR)
print('\n')
print("Original image after sharpening/resizing")
if IN_COLAB:
print("Running in Colab")
cv2_imshow(image_sharp_resize)
else:
plt.imshow(cv2.cvtColor(image_sharp_resize, cv2.COLOR_BGR2RGB))
Original image after sharpening/resizing
Observations¶
This image becomes much less distinctive once resized, even with sharpening.
image_blur = cv2.GaussianBlur(images_copy[1000],(5,5),0)
image_blur_resize = cv2.resize(image_blur, None, fx=0.50, fy=0.50, interpolation=cv2.INTER_LINEAR)
print('\n')
print("Original image after blurring/resizing")
if IN_COLAB:
print("Running in Colab")
cv2_imshow(image_blur_resize)
else:
plt.imshow(cv2.cvtColor(image_blur_resize, cv2.COLOR_BGR2RGB))
Original image after blurring/resizing
Observations¶
After blurring and resizing, this image become much softer at the edges.
Data Preparation for Modeling¶
- As we do not have many images in our dataset, we will only use 10% of our data for testing, 10% of our data for validation and 80% of our data for training.
- We will split the dataset into three parts, train,test and validation.
X_temp, X_test, y_temp, y_test = train_test_split(np.array(images_color_resized),labels , test_size=0.1, random_state=42,stratify=labels)
X_train, X_val, y_train, y_val = train_test_split(X_temp,y_temp , test_size=0.1, random_state=42,stratify=y_temp)
print(X_train.shape,y_train.shape)
print(X_val.shape,y_val.shape)
print(X_test.shape,y_test.shape)
(3847, 64, 64, 3) (3847, 1) (428, 64, 64, 3) (428, 1) (475, 64, 64, 3) (475, 1)
Encoding the target labels¶
# Convert labels from names to one hot vectors.
enc = LabelBinarizer()
y_train_encoded = enc.fit_transform(y_train)
y_val_encoded=enc.transform(y_val)
y_test_encoded=enc.transform(y_test)
y_train_encoded.shape,y_val_encoded.shape,y_test_encoded.shape
((3847, 12), (428, 12), (475, 12))
Data Normalization¶
Since the image pixel values range from 0-255, our method of normalization here will be scaling - we shall divide all the pixel values by 255 to standardize the images to have values between 0-1.
print('\n')
print("Image before normalization")
if IN_COLAB:
print("Running in Colab")
cv2_imshow(X_train[1])
else:
plt.imshow(X_train[1])
Image before normalization
# Normalizing the image pixels
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0
print('\n')
print("Image after normalization")
if IN_COLAB:
print("Running in Colab")
cv2_imshow(X_train_normalized[1])
else:
plt.imshow(X_train_normalized[1])
Image after normalization
Observations¶
- Image appears unchanged after normalization
Model Building¶
Store model results in a dataframe for later comparison.
training_metrics = pd.DataFrame(columns=["accuracy"])
test_metrics = pd.DataFrame(columns=["accuracy"])
Build first model (basic) containing multiple layers for image processing and dense layer for classification¶
CNN Model layers:¶
- Convolutional input layer, 128 feature maps with a size of 3X3 and a * rectifier activation function
- Max Pool layer with size 2×2 and a stride of 2
- Convolutional layer, 64 feature maps with a size of 3X3 and a rectifier activation function.
- Max Pool layer with size 2×2 and a stride of 2
- Convolutional layer, 32 feature maps with a size of 3X3 and a rectifier activation function.
- Max Pool layer with size 2×2 and a stride of 2
- Flatten layer
- Fully connected or Dense layers (with 16 neurons) with Relu Act.
- Dropout layer to reduce overfitting or for regularization
- Output layer with Softmax to detect multiple categories (12).
# Clearing backend
backend.clear_session()
model_name = "Base Model"
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# verify image sizes for input layer
np.shape(X_train_normalized)
(3847, 64, 64, 3)
# Since I will be trying various combos of resizing set the input layer dynamically
h, w, c = X_train_normalized[1000].shape
print('width: ', w)
print('height: ', h)
print('channel:', c)
# Intializing a sequential model
base_model = Sequential()
# create convolution and max-pooling layers with input size of images
base_model.add(Conv2D(128, (3, 3), activation='relu', padding="same", input_shape=(h, w, c)))
base_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
base_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
base_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
base_model.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
base_model.add(MaxPooling2D((2, 2), padding = 'same'))
# flatten the output of the conv layer
base_model.add(Flatten())
# add a fully connected layer
base_model.add(Dense(16, activation='relu'))
base_model.add(Dropout(0.3))
# output layer with 12 neurons and softmax
base_model.add(Dense(12, activation='softmax'))
# Adam Optimizer
opt=Adam()
# Compile the model
base_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# generate the summary
base_model.summary()
width: 64
height: 64
channel: 3
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 128) 3584
max_pooling2d (MaxPooling2 (None, 32, 32, 128) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 64) 73792
max_pooling2d_1 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_2 (Conv2D) (None, 16, 16, 32) 18464
max_pooling2d_2 (MaxPoolin (None, 8, 8, 32) 0
g2D)
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 16) 32784
dropout (Dropout) (None, 16) 0
dense_1 (Dense) (None, 12) 204
=================================================================
Total params: 128828 (503.23 KB)
Trainable params: 128828 (503.23 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Fitting the model on the train data
base_history = base_model.fit(
X_train_normalized, y_train_encoded,
epochs=50,
validation_data=(X_val_normalized,y_val_encoded),
batch_size=64,
verbose=2
)
Epoch 1/50 61/61 - 9s - loss: 2.4596 - accuracy: 0.1003 - val_loss: 2.4518 - val_accuracy: 0.1379 - 9s/epoch - 146ms/step Epoch 2/50 61/61 - 8s - loss: 2.4410 - accuracy: 0.1198 - val_loss: 2.4196 - val_accuracy: 0.1472 - 8s/epoch - 124ms/step Epoch 3/50 61/61 - 8s - loss: 2.3448 - accuracy: 0.2028 - val_loss: 2.1197 - val_accuracy: 0.3458 - 8s/epoch - 133ms/step Epoch 4/50 61/61 - 9s - loss: 2.1166 - accuracy: 0.3002 - val_loss: 1.9292 - val_accuracy: 0.3762 - 9s/epoch - 140ms/step Epoch 5/50 61/61 - 9s - loss: 1.9635 - accuracy: 0.3273 - val_loss: 1.8374 - val_accuracy: 0.3949 - 9s/epoch - 147ms/step Epoch 6/50 61/61 - 9s - loss: 1.8773 - accuracy: 0.3481 - val_loss: 1.6673 - val_accuracy: 0.4486 - 9s/epoch - 151ms/step Epoch 7/50 61/61 - 9s - loss: 1.7978 - accuracy: 0.3587 - val_loss: 1.6041 - val_accuracy: 0.4813 - 9s/epoch - 152ms/step Epoch 8/50 61/61 - 9s - loss: 1.6865 - accuracy: 0.3962 - val_loss: 1.5025 - val_accuracy: 0.5280 - 9s/epoch - 150ms/step Epoch 9/50 61/61 - 10s - loss: 1.5977 - accuracy: 0.4292 - val_loss: 1.3621 - val_accuracy: 0.5514 - 10s/epoch - 158ms/step Epoch 10/50 61/61 - 9s - loss: 1.4977 - accuracy: 0.4609 - val_loss: 1.3426 - val_accuracy: 0.5257 - 9s/epoch - 153ms/step Epoch 11/50 61/61 - 10s - loss: 1.4603 - accuracy: 0.4762 - val_loss: 1.2123 - val_accuracy: 0.5935 - 10s/epoch - 158ms/step Epoch 12/50 61/61 - 10s - loss: 1.3659 - accuracy: 0.5095 - val_loss: 1.1974 - val_accuracy: 0.6121 - 10s/epoch - 159ms/step Epoch 13/50 61/61 - 9s - loss: 1.3370 - accuracy: 0.5272 - val_loss: 1.1135 - val_accuracy: 0.6449 - 9s/epoch - 153ms/step Epoch 14/50 61/61 - 9s - loss: 1.2713 - accuracy: 0.5428 - val_loss: 1.1686 - val_accuracy: 0.6121 - 9s/epoch - 153ms/step Epoch 15/50 61/61 - 10s - loss: 1.2636 - accuracy: 0.5425 - val_loss: 1.0778 - val_accuracy: 0.6636 - 10s/epoch - 165ms/step Epoch 16/50 61/61 - 10s - loss: 1.2194 - accuracy: 0.5586 - val_loss: 1.0727 - val_accuracy: 0.6612 - 10s/epoch - 163ms/step Epoch 17/50 61/61 - 10s - loss: 1.2052 - accuracy: 0.5620 - val_loss: 1.0273 - val_accuracy: 0.6589 - 10s/epoch - 167ms/step Epoch 18/50 61/61 - 10s - loss: 1.1831 - accuracy: 0.5677 - val_loss: 1.0013 - val_accuracy: 0.6682 - 10s/epoch - 168ms/step Epoch 19/50 61/61 - 10s - loss: 1.1410 - accuracy: 0.5745 - val_loss: 1.0001 - val_accuracy: 0.6799 - 10s/epoch - 171ms/step Epoch 20/50 61/61 - 10s - loss: 1.1024 - accuracy: 0.5820 - val_loss: 0.9770 - val_accuracy: 0.6893 - 10s/epoch - 161ms/step Epoch 21/50 61/61 - 10s - loss: 1.1224 - accuracy: 0.5854 - val_loss: 1.0176 - val_accuracy: 0.6893 - 10s/epoch - 161ms/step Epoch 22/50 61/61 - 9s - loss: 1.0815 - accuracy: 0.5997 - val_loss: 1.0344 - val_accuracy: 0.6706 - 9s/epoch - 151ms/step Epoch 23/50 61/61 - 9s - loss: 1.1049 - accuracy: 0.5932 - val_loss: 1.0185 - val_accuracy: 0.6729 - 9s/epoch - 154ms/step Epoch 24/50 61/61 - 10s - loss: 1.0500 - accuracy: 0.6150 - val_loss: 0.9326 - val_accuracy: 0.7173 - 10s/epoch - 159ms/step Epoch 25/50 61/61 - 10s - loss: 1.0652 - accuracy: 0.6075 - val_loss: 0.9634 - val_accuracy: 0.7033 - 10s/epoch - 161ms/step Epoch 26/50 61/61 - 10s - loss: 1.0079 - accuracy: 0.6233 - val_loss: 0.9576 - val_accuracy: 0.6893 - 10s/epoch - 163ms/step Epoch 27/50 61/61 - 10s - loss: 1.0239 - accuracy: 0.6205 - val_loss: 0.9686 - val_accuracy: 0.6986 - 10s/epoch - 161ms/step Epoch 28/50 61/61 - 10s - loss: 0.9875 - accuracy: 0.6337 - val_loss: 0.9610 - val_accuracy: 0.6916 - 10s/epoch - 162ms/step Epoch 29/50 61/61 - 10s - loss: 0.9913 - accuracy: 0.6265 - val_loss: 0.9179 - val_accuracy: 0.7173 - 10s/epoch - 166ms/step Epoch 30/50 61/61 - 10s - loss: 0.9607 - accuracy: 0.6470 - val_loss: 0.9161 - val_accuracy: 0.7173 - 10s/epoch - 168ms/step Epoch 31/50 61/61 - 10s - loss: 0.9633 - accuracy: 0.6408 - val_loss: 0.9218 - val_accuracy: 0.7173 - 10s/epoch - 166ms/step Epoch 32/50 61/61 - 10s - loss: 0.9297 - accuracy: 0.6512 - val_loss: 0.9686 - val_accuracy: 0.6939 - 10s/epoch - 172ms/step Epoch 33/50 61/61 - 10s - loss: 0.9535 - accuracy: 0.6488 - val_loss: 0.9245 - val_accuracy: 0.7009 - 10s/epoch - 166ms/step Epoch 34/50 61/61 - 10s - loss: 0.9448 - accuracy: 0.6501 - val_loss: 0.9507 - val_accuracy: 0.7336 - 10s/epoch - 163ms/step Epoch 35/50 61/61 - 10s - loss: 0.8948 - accuracy: 0.6597 - val_loss: 0.9495 - val_accuracy: 0.7196 - 10s/epoch - 166ms/step Epoch 36/50 61/61 - 10s - loss: 0.9174 - accuracy: 0.6545 - val_loss: 0.9571 - val_accuracy: 0.7103 - 10s/epoch - 164ms/step Epoch 37/50 61/61 - 10s - loss: 0.8911 - accuracy: 0.6642 - val_loss: 0.9312 - val_accuracy: 0.7313 - 10s/epoch - 163ms/step Epoch 38/50 61/61 - 10s - loss: 0.8593 - accuracy: 0.6772 - val_loss: 1.0048 - val_accuracy: 0.7196 - 10s/epoch - 167ms/step Epoch 39/50 61/61 - 10s - loss: 0.8419 - accuracy: 0.6852 - val_loss: 0.9205 - val_accuracy: 0.7313 - 10s/epoch - 159ms/step Epoch 40/50 61/61 - 9s - loss: 0.8509 - accuracy: 0.6759 - val_loss: 0.9675 - val_accuracy: 0.7173 - 9s/epoch - 155ms/step Epoch 41/50 61/61 - 10s - loss: 0.8745 - accuracy: 0.6740 - val_loss: 1.0010 - val_accuracy: 0.7243 - 10s/epoch - 161ms/step Epoch 42/50 61/61 - 10s - loss: 0.8531 - accuracy: 0.6777 - val_loss: 0.9933 - val_accuracy: 0.7220 - 10s/epoch - 166ms/step Epoch 43/50 61/61 - 10s - loss: 0.8445 - accuracy: 0.6860 - val_loss: 0.9724 - val_accuracy: 0.7150 - 10s/epoch - 166ms/step Epoch 44/50 61/61 - 10s - loss: 0.8401 - accuracy: 0.6798 - val_loss: 0.9318 - val_accuracy: 0.7407 - 10s/epoch - 169ms/step Epoch 45/50 61/61 - 11s - loss: 0.8392 - accuracy: 0.6831 - val_loss: 0.9313 - val_accuracy: 0.7570 - 11s/epoch - 174ms/step Epoch 46/50 61/61 - 10s - loss: 0.8424 - accuracy: 0.6805 - val_loss: 0.9006 - val_accuracy: 0.7640 - 10s/epoch - 162ms/step Epoch 47/50 61/61 - 10s - loss: 0.8013 - accuracy: 0.6972 - val_loss: 0.9501 - val_accuracy: 0.7383 - 10s/epoch - 160ms/step Epoch 48/50 61/61 - 10s - loss: 0.8121 - accuracy: 0.6870 - val_loss: 1.0440 - val_accuracy: 0.7220 - 10s/epoch - 160ms/step Epoch 49/50 61/61 - 10s - loss: 0.8200 - accuracy: 0.6852 - val_loss: 0.9955 - val_accuracy: 0.7243 - 10s/epoch - 167ms/step Epoch 50/50 61/61 - 10s - loss: 0.7776 - accuracy: 0.7000 - val_loss: 0.9834 - val_accuracy: 0.7336 - 10s/epoch - 165ms/step
Model Evaluation
plt.plot(base_history.history['accuracy'])
plt.plot(base_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- Fom the graph we can see that the training accuracy of the model was reached the 70% range and the validation accuracy was divergent and eventually reached the 60% range.
- It does not seem to improve much past epoch 25.
Loss
plt.plot(base_history.history['loss'])
plt.plot(base_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- Loss between Training and validation swaps around epoch 34.
- It does not seem to improve much past epoch 35.
Evaluate the model on test data
base_history_accuracy = base_model.evaluate(X_test_normalized,
y_test_encoded,
#verbose=2
)
15/15 [==============================] - 0s 27ms/step - loss: 0.9205 - accuracy: 0.7326
best_model_accuracy = base_history.history['accuracy'][np.argmin(base_history.history['loss'])]
best_model_accuracy
0.7000259757041931
#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = base_history_accuracy[1]
print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])
accuracy 0.700026 Name: Base Model, dtype: float64 accuracy 0.732632 Name: Base Model, dtype: float64
Plotting the Confusion Matrix
# Here we would get the output as probablities for each category
y_pred=base_model.predict(X_test_normalized)
15/15 [==============================] - 0s 28ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Observations¶
- We can observe that some of the plant classes are not predicted correctly.
- It appears none of the Black-grass samples were correctly identified (23, 3).
- We can also observe that Black-grass(26), Common Chickweed(12), Common Wheat(15), Maize(10), Scentless Mayweed(11), Sheperds Purse(10) and Fat Hen(9) have the largest misclassifications.
Complex CNN model containing multiple layers for image processing and dense layer for classification¶
Here we will add complexity to the model to try and get better results.
CNN Model layers:¶
- Convolutional input layer, 32 feature maps with a size of 3X3 and a * rectifier activation function
- Max Pool layer with size 2×2 and a stride of 2
- Convolutional layer, 64 feature maps with a size of 3X3 and a rectifier activation function.
- Max Pool layer with size 2×2 and a stride of 2
- Convolutional layer, 128 feature maps with a size of 3X3 and a rectifier activation function.
- Max Pool layer with size 2×2 and a stride of 2
- Convolutional layer, 128 feature maps with a size of 3X3 and a rectifier activation function.
- Max Pool layer with size 2×2 and a stride of 2
- Flatten layer
- Fully connected or Dense layers (with 128 neurons) with Relu Act.
- Dropout layer to reduce overfitting or for regularization
- Fully connected or Dense layers (with 512 neurons) with Relu Act.
- Dropout layer to reduce overfitting or for regularization
- Fully connected or Dense layers (with 256 neurons) with Relu Act.
- Dropout layer to reduce overfitting or for regularization
- Output layer with Softmax to detect multiple categories (12).
# Clearing backend
backend.clear_session()
model_name = "Complex Model"
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# verify image sizes for input layer
np.shape(X_train_normalized)
(3847, 64, 64, 3)
h, w, c = X_train_normalized[1000].shape
print('width: ', w)
print('height: ', h)
print('channel:', c)
# Intializing a sequential model
complex_model = Sequential()
# create convolution and max-pooling layers with input size of images
complex_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h, w, c)))
complex_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
complex_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
complex_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
complex_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
complex_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model.add(MaxPooling2D((2, 2), padding = 'same'))
# flatten the output of the conv layer
complex_model.add(Flatten())
# add a fully connected layer
complex_model.add(Dense(128, activation='relu'))
complex_model.add(Dropout(0.3))
# add a fully connected layer
complex_model.add(Dense(512, activation='relu'))
complex_model.add(Dropout(0.3))
# add a fully connected layer
complex_model.add(Dense(256, activation='relu'))
complex_model.add(Dropout(0.3))
# output layer with 12 neurons and softmax
complex_model.add(Dense(12, activation='softmax'))
# Adam Optimizer
opt=Adam()
# Compile the model
complex_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# generate the summary
complex_model.summary()
width: 64
height: 64
channel: 3
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 32) 896
max_pooling2d (MaxPooling2 (None, 32, 32, 32) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 64) 18496
max_pooling2d_1 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_2 (Conv2D) (None, 16, 16, 128) 73856
max_pooling2d_2 (MaxPoolin (None, 8, 8, 128) 0
g2D)
conv2d_3 (Conv2D) (None, 8, 8, 128) 147584
max_pooling2d_3 (MaxPoolin (None, 4, 4, 128) 0
g2D)
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 128) 262272
dropout (Dropout) (None, 128) 0
dense_1 (Dense) (None, 512) 66048
dropout_1 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 256) 131328
dropout_2 (Dropout) (None, 256) 0
dense_3 (Dense) (None, 12) 3084
=================================================================
Total params: 703564 (2.68 MB)
Trainable params: 703564 (2.68 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Fitting the model on the train data
complex_history = complex_model.fit(
X_train_normalized, y_train_encoded,
epochs=50,
validation_data=(X_val_normalized,y_val_encoded),
batch_size=64,
verbose=2
)
Epoch 1/50 61/61 - 7s - loss: 2.4347 - accuracy: 0.1318 - val_loss: 2.4040 - val_accuracy: 0.1379 - 7s/epoch - 113ms/step Epoch 2/50 61/61 - 5s - loss: 2.3157 - accuracy: 0.1994 - val_loss: 1.9667 - val_accuracy: 0.2757 - 5s/epoch - 89ms/step Epoch 3/50 61/61 - 6s - loss: 1.8033 - accuracy: 0.3384 - val_loss: 1.5833 - val_accuracy: 0.4299 - 6s/epoch - 93ms/step Epoch 4/50 61/61 - 5s - loss: 1.5718 - accuracy: 0.4359 - val_loss: 1.4064 - val_accuracy: 0.4813 - 5s/epoch - 89ms/step Epoch 5/50 61/61 - 6s - loss: 1.3404 - accuracy: 0.5269 - val_loss: 1.3290 - val_accuracy: 0.5444 - 6s/epoch - 98ms/step Epoch 6/50 61/61 - 6s - loss: 1.2357 - accuracy: 0.5719 - val_loss: 1.1874 - val_accuracy: 0.6145 - 6s/epoch - 99ms/step Epoch 7/50 61/61 - 6s - loss: 1.0727 - accuracy: 0.6192 - val_loss: 1.0276 - val_accuracy: 0.6402 - 6s/epoch - 96ms/step Epoch 8/50 61/61 - 6s - loss: 1.0283 - accuracy: 0.6371 - val_loss: 1.0235 - val_accuracy: 0.6495 - 6s/epoch - 97ms/step Epoch 9/50 61/61 - 6s - loss: 0.8854 - accuracy: 0.6875 - val_loss: 0.8745 - val_accuracy: 0.7103 - 6s/epoch - 98ms/step Epoch 10/50 61/61 - 6s - loss: 0.8156 - accuracy: 0.7070 - val_loss: 0.8217 - val_accuracy: 0.7196 - 6s/epoch - 100ms/step Epoch 11/50 61/61 - 6s - loss: 0.7527 - accuracy: 0.7325 - val_loss: 0.8224 - val_accuracy: 0.7430 - 6s/epoch - 100ms/step Epoch 12/50 61/61 - 6s - loss: 0.6732 - accuracy: 0.7637 - val_loss: 0.7062 - val_accuracy: 0.7757 - 6s/epoch - 105ms/step Epoch 13/50 61/61 - 6s - loss: 0.6550 - accuracy: 0.7702 - val_loss: 0.6765 - val_accuracy: 0.7780 - 6s/epoch - 102ms/step Epoch 14/50 61/61 - 6s - loss: 0.5796 - accuracy: 0.7954 - val_loss: 0.7914 - val_accuracy: 0.7570 - 6s/epoch - 101ms/step Epoch 15/50 61/61 - 6s - loss: 0.5378 - accuracy: 0.8144 - val_loss: 0.7139 - val_accuracy: 0.7804 - 6s/epoch - 100ms/step Epoch 16/50 61/61 - 6s - loss: 0.4865 - accuracy: 0.8253 - val_loss: 0.8664 - val_accuracy: 0.7617 - 6s/epoch - 98ms/step Epoch 17/50 61/61 - 6s - loss: 0.5135 - accuracy: 0.8217 - val_loss: 0.7266 - val_accuracy: 0.7874 - 6s/epoch - 99ms/step Epoch 18/50 61/61 - 6s - loss: 0.4516 - accuracy: 0.8407 - val_loss: 0.7053 - val_accuracy: 0.8061 - 6s/epoch - 98ms/step Epoch 19/50 61/61 - 6s - loss: 0.3818 - accuracy: 0.8646 - val_loss: 0.7122 - val_accuracy: 0.8037 - 6s/epoch - 99ms/step Epoch 20/50 61/61 - 6s - loss: 0.3534 - accuracy: 0.8734 - val_loss: 0.9190 - val_accuracy: 0.7500 - 6s/epoch - 104ms/step Epoch 21/50 61/61 - 6s - loss: 0.4036 - accuracy: 0.8568 - val_loss: 0.5883 - val_accuracy: 0.8528 - 6s/epoch - 101ms/step Epoch 22/50 61/61 - 6s - loss: 0.3096 - accuracy: 0.8906 - val_loss: 0.6213 - val_accuracy: 0.8224 - 6s/epoch - 101ms/step Epoch 23/50 61/61 - 6s - loss: 0.2988 - accuracy: 0.8924 - val_loss: 0.5923 - val_accuracy: 0.8388 - 6s/epoch - 97ms/step Epoch 24/50 61/61 - 6s - loss: 0.2919 - accuracy: 0.8984 - val_loss: 0.6838 - val_accuracy: 0.8294 - 6s/epoch - 101ms/step Epoch 25/50 61/61 - 6s - loss: 0.2891 - accuracy: 0.8916 - val_loss: 0.6880 - val_accuracy: 0.8318 - 6s/epoch - 94ms/step Epoch 26/50 61/61 - 6s - loss: 0.2444 - accuracy: 0.9124 - val_loss: 0.6306 - val_accuracy: 0.8458 - 6s/epoch - 98ms/step Epoch 27/50 61/61 - 6s - loss: 0.2725 - accuracy: 0.9038 - val_loss: 0.6195 - val_accuracy: 0.8411 - 6s/epoch - 93ms/step Epoch 28/50 61/61 - 6s - loss: 0.2339 - accuracy: 0.9153 - val_loss: 0.7309 - val_accuracy: 0.8107 - 6s/epoch - 105ms/step Epoch 29/50 61/61 - 7s - loss: 0.2325 - accuracy: 0.9121 - val_loss: 0.6667 - val_accuracy: 0.8435 - 7s/epoch - 108ms/step Epoch 30/50 61/61 - 6s - loss: 0.2069 - accuracy: 0.9218 - val_loss: 0.7933 - val_accuracy: 0.8107 - 6s/epoch - 103ms/step Epoch 31/50 61/61 - 6s - loss: 0.2072 - accuracy: 0.9259 - val_loss: 0.7165 - val_accuracy: 0.8411 - 6s/epoch - 99ms/step Epoch 32/50 61/61 - 6s - loss: 0.1889 - accuracy: 0.9288 - val_loss: 0.6839 - val_accuracy: 0.8435 - 6s/epoch - 104ms/step Epoch 33/50 61/61 - 6s - loss: 0.1845 - accuracy: 0.9311 - val_loss: 0.9491 - val_accuracy: 0.8154 - 6s/epoch - 101ms/step Epoch 34/50 61/61 - 6s - loss: 0.2155 - accuracy: 0.9257 - val_loss: 0.6805 - val_accuracy: 0.8388 - 6s/epoch - 95ms/step Epoch 35/50 61/61 - 6s - loss: 0.1635 - accuracy: 0.9392 - val_loss: 0.7970 - val_accuracy: 0.8224 - 6s/epoch - 97ms/step Epoch 36/50 61/61 - 6s - loss: 0.1609 - accuracy: 0.9446 - val_loss: 0.7326 - val_accuracy: 0.8411 - 6s/epoch - 104ms/step Epoch 37/50 61/61 - 6s - loss: 0.1809 - accuracy: 0.9376 - val_loss: 0.7756 - val_accuracy: 0.8294 - 6s/epoch - 104ms/step Epoch 38/50 61/61 - 6s - loss: 0.1577 - accuracy: 0.9459 - val_loss: 0.7072 - val_accuracy: 0.8341 - 6s/epoch - 103ms/step Epoch 39/50 61/61 - 6s - loss: 0.1201 - accuracy: 0.9594 - val_loss: 0.7259 - val_accuracy: 0.8435 - 6s/epoch - 101ms/step Epoch 40/50 61/61 - 6s - loss: 0.1136 - accuracy: 0.9584 - val_loss: 0.8301 - val_accuracy: 0.8294 - 6s/epoch - 92ms/step Epoch 41/50 61/61 - 6s - loss: 0.1585 - accuracy: 0.9465 - val_loss: 0.9825 - val_accuracy: 0.8107 - 6s/epoch - 93ms/step Epoch 42/50 61/61 - 6s - loss: 0.1818 - accuracy: 0.9363 - val_loss: 0.7074 - val_accuracy: 0.8528 - 6s/epoch - 96ms/step Epoch 43/50 61/61 - 6s - loss: 0.1364 - accuracy: 0.9535 - val_loss: 0.8272 - val_accuracy: 0.8411 - 6s/epoch - 99ms/step Epoch 44/50 61/61 - 6s - loss: 0.1742 - accuracy: 0.9441 - val_loss: 0.7004 - val_accuracy: 0.8551 - 6s/epoch - 100ms/step Epoch 45/50 61/61 - 6s - loss: 0.1075 - accuracy: 0.9639 - val_loss: 0.7788 - val_accuracy: 0.8598 - 6s/epoch - 105ms/step Epoch 46/50 61/61 - 6s - loss: 0.1207 - accuracy: 0.9576 - val_loss: 0.6841 - val_accuracy: 0.8575 - 6s/epoch - 102ms/step Epoch 47/50 61/61 - 7s - loss: 0.0969 - accuracy: 0.9665 - val_loss: 0.7042 - val_accuracy: 0.8388 - 7s/epoch - 108ms/step Epoch 48/50 61/61 - 6s - loss: 0.0831 - accuracy: 0.9711 - val_loss: 0.6746 - val_accuracy: 0.8505 - 6s/epoch - 103ms/step Epoch 49/50 61/61 - 6s - loss: 0.0955 - accuracy: 0.9670 - val_loss: 0.9756 - val_accuracy: 0.8364 - 6s/epoch - 99ms/step Epoch 50/50 61/61 - 7s - loss: 0.1566 - accuracy: 0.9504 - val_loss: 0.6661 - val_accuracy: 0.8715 - 7s/epoch - 110ms/step
Model Evaluation
plt.plot(complex_history.history['accuracy'])
plt.plot(complex_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- Fom the graph we can see that the training accuracy of the model was improved over the base model and the validation accuracy was also improved.
- The model seems to overfit on the training data.
- It does not seem to improve much past epoch 25. We will use 25 as our base epochs.
Loss
plt.plot(complex_history.history['loss'])
plt.plot(complex_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- Loss between Training and validation diverges around epoch 12.
- Validation loss holds fairly constant.
- It does not seem to improve much past epoch 25.
Evaluate the model on test data
complex_history_accuracy = complex_model.evaluate(X_test_normalized,
y_test_encoded,
#verbose=2
)
15/15 [==============================] - 1s 38ms/step - loss: 0.7007 - accuracy: 0.8211
best_model_accuracy = complex_history.history['accuracy'][np.argmin(complex_history.history['loss'])]
best_model_accuracy
0.9711463451385498
#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = complex_history_accuracy[1]
print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])
accuracy 0.971146 Name: Complex Model, dtype: float64 accuracy 0.821053 Name: Complex Model, dtype: float64
Observations¶
- Test Accuracy is 82%
- Training model accuracy with least loss is 97%, showing overfitting.
Plotting the Confusion Matrix
# Here we would get the output as probablities for each category
y_pred=complex_model.predict(X_test_normalized)
15/15 [==============================] - 1s 39ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Observations¶
- We can observe that some of the plant classes are still not predicted correctly.
- It appears that we now have some of the Black-grass samples correctly identified.
- Common Chickweed has dropped from 12 misidentified to 3.
- Common Wheat has dropped from 15 misidentified to 6.
- Fat Hen has dropped from 9 misidentified to 8.
- Maize has dropped from 10 misidentified to 3.
- Sheperds Purse has dropped from 10 misidentified to 5.
- Scentless Mayweed has increased from 11 misidentified to 12.
- Loose Silky-bent remains 5, but it is now misidentified differently.
- Black-grass(22), Common Wheat and Scentless Mayweed have the largest misclassifications.
Model Performance Improvement¶
Applying Class weights to complex model¶
from sklearn.utils import class_weight
labelList = np.unique(labels)
class_weights = class_weight.compute_class_weight(class_weight = "balanced",
classes = np.array(labelList),
y = y_train.values.reshape(-1)
)
class_weights = dict(zip(np.array(range(len(labelList))), class_weights))
#print calculated class weights
class_weights
{0: 1.5050860719874803,
1: 1.0145042194092826,
2: 1.3818247126436782,
3: 0.6476430976430977,
4: 1.7909683426443204,
5: 0.8348524305555556,
6: 0.604874213836478,
7: 1.7909683426443204,
8: 0.7669457735247209,
9: 1.714349376114082,
10: 0.7974709784411277,
11: 1.0275106837606838}
# Clearing backend
backend.clear_session()
model_name = "Complex Model with Class Weights applied"
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# verify image sizes for input layer
np.shape(X_train_normalized)
(3847, 64, 64, 3)
h, w, c = X_train_normalized[1000].shape
print('width: ', w)
print('height: ', h)
print('channel:', c)
# Intializing a sequential model
complex_model_class_weights = Sequential()
# create convolution and max-pooling layers with input size of images
complex_model_class_weights.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h, w, c)))
complex_model_class_weights.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
complex_model_class_weights.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
complex_model_class_weights.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
complex_model_class_weights.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model_class_weights.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
complex_model_class_weights.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model_class_weights.add(MaxPooling2D((2, 2), padding = 'same'))
# flatten the output of the conv layer
complex_model_class_weights.add(Flatten())
# add a fully connected layer
complex_model_class_weights.add(Dense(128, activation='relu'))
complex_model_class_weights.add(Dropout(0.3))
# add a fully connected layer
complex_model_class_weights.add(Dense(512, activation='relu'))
complex_model_class_weights.add(Dropout(0.3))
# add a fully connected layer
complex_model_class_weights.add(Dense(256, activation='relu'))
complex_model_class_weights.add(Dropout(0.3))
# output layer with 12 neurons and softmax
complex_model_class_weights.add(Dense(12, activation='softmax'))
# Adam Optimizer
opt=Adam()
# Compile the model
complex_model_class_weights.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# generate the summary
complex_model_class_weights.summary()
width: 64
height: 64
channel: 3
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 32) 896
max_pooling2d (MaxPooling2 (None, 32, 32, 32) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 64) 18496
max_pooling2d_1 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_2 (Conv2D) (None, 16, 16, 128) 73856
max_pooling2d_2 (MaxPoolin (None, 8, 8, 128) 0
g2D)
conv2d_3 (Conv2D) (None, 8, 8, 128) 147584
max_pooling2d_3 (MaxPoolin (None, 4, 4, 128) 0
g2D)
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 128) 262272
dropout (Dropout) (None, 128) 0
dense_1 (Dense) (None, 512) 66048
dropout_1 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 256) 131328
dropout_2 (Dropout) (None, 256) 0
dense_3 (Dense) (None, 12) 3084
=================================================================
Total params: 703564 (2.68 MB)
Trainable params: 703564 (2.68 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Fitting the model on the train data
complex_class_weights_history = complex_model_class_weights.fit(
X_train_normalized, y_train_encoded,
class_weight=class_weights,
epochs=50,
validation_data=(X_val_normalized,y_val_encoded),
batch_size=64,
verbose=2
)
Epoch 1/50 61/61 - 8s - loss: 2.4733 - accuracy: 0.0894 - val_loss: 2.4177 - val_accuracy: 0.1215 - 8s/epoch - 132ms/step Epoch 2/50 61/61 - 6s - loss: 2.1632 - accuracy: 0.1734 - val_loss: 1.9372 - val_accuracy: 0.2383 - 6s/epoch - 100ms/step Epoch 3/50 61/61 - 6s - loss: 1.8823 - accuracy: 0.2727 - val_loss: 1.7028 - val_accuracy: 0.2664 - 6s/epoch - 101ms/step Epoch 4/50 61/61 - 6s - loss: 1.7305 - accuracy: 0.3140 - val_loss: 1.7540 - val_accuracy: 0.3271 - 6s/epoch - 102ms/step Epoch 5/50 61/61 - 6s - loss: 1.7018 - accuracy: 0.3267 - val_loss: 1.5707 - val_accuracy: 0.3949 - 6s/epoch - 99ms/step Epoch 6/50 61/61 - 6s - loss: 1.5672 - accuracy: 0.3800 - val_loss: 1.5150 - val_accuracy: 0.4346 - 6s/epoch - 98ms/step Epoch 7/50 61/61 - 6s - loss: 1.5019 - accuracy: 0.4079 - val_loss: 1.3537 - val_accuracy: 0.5070 - 6s/epoch - 97ms/step Epoch 8/50 61/61 - 6s - loss: 1.4124 - accuracy: 0.4713 - val_loss: 1.2559 - val_accuracy: 0.5678 - 6s/epoch - 97ms/step Epoch 9/50 61/61 - 6s - loss: 1.2504 - accuracy: 0.5292 - val_loss: 1.1399 - val_accuracy: 0.5607 - 6s/epoch - 94ms/step Epoch 10/50 61/61 - 6s - loss: 1.1569 - accuracy: 0.5584 - val_loss: 1.0578 - val_accuracy: 0.6565 - 6s/epoch - 92ms/step Epoch 11/50 61/61 - 6s - loss: 1.0386 - accuracy: 0.6239 - val_loss: 0.9669 - val_accuracy: 0.6752 - 6s/epoch - 91ms/step Epoch 12/50 61/61 - 6s - loss: 0.9406 - accuracy: 0.6501 - val_loss: 1.1948 - val_accuracy: 0.5888 - 6s/epoch - 93ms/step Epoch 13/50 61/61 - 6s - loss: 0.8967 - accuracy: 0.6623 - val_loss: 0.7890 - val_accuracy: 0.7617 - 6s/epoch - 99ms/step Epoch 14/50 61/61 - 6s - loss: 0.7777 - accuracy: 0.7057 - val_loss: 0.9701 - val_accuracy: 0.6799 - 6s/epoch - 101ms/step Epoch 15/50 61/61 - 6s - loss: 0.7679 - accuracy: 0.7159 - val_loss: 0.8432 - val_accuracy: 0.7336 - 6s/epoch - 98ms/step Epoch 16/50 61/61 - 6s - loss: 0.6790 - accuracy: 0.7421 - val_loss: 1.0318 - val_accuracy: 0.6822 - 6s/epoch - 106ms/step Epoch 17/50 61/61 - 6s - loss: 0.6680 - accuracy: 0.7502 - val_loss: 0.8275 - val_accuracy: 0.7453 - 6s/epoch - 100ms/step Epoch 18/50 61/61 - 6s - loss: 0.6283 - accuracy: 0.7549 - val_loss: 0.7667 - val_accuracy: 0.7757 - 6s/epoch - 97ms/step Epoch 19/50 61/61 - 6s - loss: 0.5610 - accuracy: 0.7796 - val_loss: 0.7019 - val_accuracy: 0.7921 - 6s/epoch - 98ms/step Epoch 20/50 61/61 - 6s - loss: 0.5165 - accuracy: 0.8001 - val_loss: 0.7419 - val_accuracy: 0.7780 - 6s/epoch - 95ms/step Epoch 21/50 61/61 - 6s - loss: 0.4671 - accuracy: 0.8209 - val_loss: 0.8504 - val_accuracy: 0.7313 - 6s/epoch - 93ms/step Epoch 22/50 61/61 - 6s - loss: 0.4751 - accuracy: 0.8175 - val_loss: 0.9202 - val_accuracy: 0.7383 - 6s/epoch - 93ms/step Epoch 23/50 61/61 - 6s - loss: 0.4279 - accuracy: 0.8339 - val_loss: 0.8281 - val_accuracy: 0.7383 - 6s/epoch - 91ms/step Epoch 24/50 61/61 - 6s - loss: 0.4244 - accuracy: 0.8225 - val_loss: 0.7795 - val_accuracy: 0.7664 - 6s/epoch - 95ms/step Epoch 25/50 61/61 - 6s - loss: 0.4738 - accuracy: 0.8173 - val_loss: 0.7374 - val_accuracy: 0.8014 - 6s/epoch - 94ms/step Epoch 26/50 61/61 - 6s - loss: 0.4435 - accuracy: 0.8355 - val_loss: 0.8845 - val_accuracy: 0.7523 - 6s/epoch - 98ms/step Epoch 27/50 61/61 - 6s - loss: 0.3591 - accuracy: 0.8544 - val_loss: 0.7173 - val_accuracy: 0.7921 - 6s/epoch - 98ms/step Epoch 28/50 61/61 - 6s - loss: 0.3211 - accuracy: 0.8713 - val_loss: 0.7383 - val_accuracy: 0.7967 - 6s/epoch - 100ms/step Epoch 29/50 61/61 - 6s - loss: 0.2918 - accuracy: 0.8846 - val_loss: 0.7289 - val_accuracy: 0.8131 - 6s/epoch - 97ms/step Epoch 30/50 61/61 - 6s - loss: 0.3121 - accuracy: 0.8783 - val_loss: 0.7548 - val_accuracy: 0.7944 - 6s/epoch - 101ms/step Epoch 31/50 61/61 - 6s - loss: 0.3089 - accuracy: 0.8783 - val_loss: 0.6678 - val_accuracy: 0.8224 - 6s/epoch - 95ms/step Epoch 32/50 61/61 - 6s - loss: 0.2557 - accuracy: 0.8976 - val_loss: 0.9462 - val_accuracy: 0.7640 - 6s/epoch - 93ms/step Epoch 33/50 61/61 - 6s - loss: 0.2774 - accuracy: 0.8900 - val_loss: 0.8480 - val_accuracy: 0.7827 - 6s/epoch - 101ms/step Epoch 34/50 61/61 - 6s - loss: 0.2505 - accuracy: 0.8968 - val_loss: 0.7831 - val_accuracy: 0.8084 - 6s/epoch - 102ms/step Epoch 35/50 61/61 - 6s - loss: 0.2694 - accuracy: 0.8950 - val_loss: 0.7323 - val_accuracy: 0.8154 - 6s/epoch - 99ms/step Epoch 36/50 61/61 - 6s - loss: 0.2176 - accuracy: 0.9101 - val_loss: 0.7685 - val_accuracy: 0.8107 - 6s/epoch - 98ms/step Epoch 37/50 61/61 - 6s - loss: 0.2277 - accuracy: 0.9129 - val_loss: 0.7944 - val_accuracy: 0.7991 - 6s/epoch - 100ms/step Epoch 38/50 61/61 - 6s - loss: 0.2324 - accuracy: 0.8929 - val_loss: 0.7791 - val_accuracy: 0.8201 - 6s/epoch - 100ms/step Epoch 39/50 61/61 - 6s - loss: 0.2752 - accuracy: 0.8890 - val_loss: 0.8073 - val_accuracy: 0.8061 - 6s/epoch - 95ms/step Epoch 40/50 61/61 - 6s - loss: 0.2179 - accuracy: 0.9095 - val_loss: 0.7444 - val_accuracy: 0.8318 - 6s/epoch - 96ms/step Epoch 41/50 61/61 - 6s - loss: 0.2212 - accuracy: 0.9132 - val_loss: 0.8191 - val_accuracy: 0.8131 - 6s/epoch - 98ms/step Epoch 42/50 61/61 - 6s - loss: 0.1954 - accuracy: 0.9186 - val_loss: 0.7774 - val_accuracy: 0.8294 - 6s/epoch - 93ms/step Epoch 43/50 61/61 - 6s - loss: 0.1977 - accuracy: 0.9283 - val_loss: 0.7761 - val_accuracy: 0.7991 - 6s/epoch - 92ms/step Epoch 44/50 61/61 - 6s - loss: 0.1874 - accuracy: 0.9238 - val_loss: 0.8406 - val_accuracy: 0.8107 - 6s/epoch - 96ms/step Epoch 45/50 61/61 - 6s - loss: 0.1646 - accuracy: 0.9329 - val_loss: 0.7810 - val_accuracy: 0.8294 - 6s/epoch - 94ms/step Epoch 46/50 61/61 - 6s - loss: 0.1658 - accuracy: 0.9363 - val_loss: 0.8834 - val_accuracy: 0.7944 - 6s/epoch - 92ms/step Epoch 47/50 61/61 - 6s - loss: 0.1659 - accuracy: 0.9288 - val_loss: 0.9497 - val_accuracy: 0.8224 - 6s/epoch - 96ms/step Epoch 48/50 61/61 - 6s - loss: 0.1735 - accuracy: 0.9303 - val_loss: 0.7504 - val_accuracy: 0.8178 - 6s/epoch - 96ms/step Epoch 49/50 61/61 - 6s - loss: 0.1488 - accuracy: 0.9392 - val_loss: 0.8148 - val_accuracy: 0.8178 - 6s/epoch - 93ms/step Epoch 50/50 61/61 - 6s - loss: 0.1673 - accuracy: 0.9392 - val_loss: 0.8270 - val_accuracy: 0.8154 - 6s/epoch - 99ms/step
Model Evaluation
plt.plot(complex_class_weights_history.history['accuracy'])
plt.plot(complex_class_weights_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- Fom the graph we can see that the training accuracy of the model was improved over the base model and the validation accuracy was also improved.
- The model seems to overfit on the training data.
- It does not seem to improve much past epoch 25. We will use 25 as our base epochs.
Loss
plt.plot(complex_class_weights_history.history['loss'])
plt.plot(complex_class_weights_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- Loss between Training and validation diverges around epoch 12.
- Validation loss holds fairly constant.
- It does not seem to improve much past epoch 25.
Evaluate the model on test data
complex_class_weights_history_accuracy = complex_model_class_weights.evaluate(X_test_normalized,
y_test_encoded,
#verbose=2
)
15/15 [==============================] - 1s 39ms/step - loss: 0.8297 - accuracy: 0.7958
best_model_accuracy = complex_class_weights_history.history['accuracy'][np.argmin(complex_class_weights_history.history['loss'])]
best_model_accuracy
0.9391734004020691
#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = complex_class_weights_history_accuracy[1]
print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])
accuracy 0.939173 Name: Complex Model with Class Weights applied, dtype: float64 accuracy 0.795789 Name: Complex Model with Class Weights applied, dtype: float64
Observations¶
- Test Accuracy is 80%
- Training model accuracy with least loss is 94%.
- Adjusting the class weights did not improve identification.
Plotting the Confusion Matrix
# Here we would get the output as probablities for each category
y_pred=complex_model.predict(X_test_normalized)
15/15 [==============================] - 1s 40ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Observations¶
- The misidentified plants did not improve much, if at all. We will try another strategy.
Model using adjustment of the Adam optimizer:¶
In this model we will not use Class weights. We will adjust the adam optimizer using learning rate, decay rate and avoiding division by zero.
# Clearing backend
backend.clear_session()
model_name = "Complex Model with Adam optimizer adjusted"
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# verify image sizes for input layer
np.shape(X_train_normalized)
(3847, 64, 64, 3)
h, w, c = X_train_normalized[1000].shape
print('width: ', w)
print('height: ', h)
print('channel:', c)
# Intializing a sequential model
complex_model_adam_opt = Sequential()
# create convolution and max-pooling layers with input size of images
complex_model_adam_opt.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h, w, c)))
complex_model_adam_opt.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
complex_model_adam_opt.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
complex_model_adam_opt.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
complex_model_adam_opt.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model_adam_opt.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
complex_model_adam_opt.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
complex_model_adam_opt.add(MaxPooling2D((2, 2), padding = 'same'))
# flatten the output of the conv layer
complex_model_adam_opt.add(Flatten())
# add a fully connected layer
complex_model_adam_opt.add(Dense(128, activation='relu'))
complex_model_adam_opt.add(Dropout(0.3))
# add a fully connected layer
complex_model_adam_opt.add(Dense(512, activation='relu'))
complex_model_adam_opt.add(Dropout(0.3))
# add a fully connected layer
complex_model_adam_opt.add(Dense(256, activation='relu'))
complex_model_adam_opt.add(Dropout(0.3))
# output layer with 12 neurons and softmax
complex_model_adam_opt.add(Dense(12, activation='softmax'))
# Adam Optimizer
adam_opt = optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)
# Compile the model
complex_model_adam_opt.compile(optimizer=adam_opt, loss='categorical_crossentropy', metrics=['accuracy'])
# generate the summary
complex_model_adam_opt.summary()
width: 64
height: 64
channel: 3
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 32) 896
max_pooling2d (MaxPooling2 (None, 32, 32, 32) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 64) 18496
max_pooling2d_1 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_2 (Conv2D) (None, 16, 16, 128) 73856
max_pooling2d_2 (MaxPoolin (None, 8, 8, 128) 0
g2D)
conv2d_3 (Conv2D) (None, 8, 8, 128) 147584
max_pooling2d_3 (MaxPoolin (None, 4, 4, 128) 0
g2D)
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 128) 262272
dropout (Dropout) (None, 128) 0
dense_1 (Dense) (None, 512) 66048
dropout_1 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 256) 131328
dropout_2 (Dropout) (None, 256) 0
dense_3 (Dense) (None, 12) 3084
=================================================================
Total params: 703564 (2.68 MB)
Trainable params: 703564 (2.68 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Fitting the model on the train data
complex_adam_opt_history = complex_model_adam_opt.fit(
X_train_normalized, y_train_encoded,
epochs=50,
validation_data=(X_val_normalized,y_val_encoded),
batch_size=64,
verbose=2
)
Epoch 1/50 61/61 - 7s - loss: 2.4335 - accuracy: 0.1300 - val_loss: 2.4119 - val_accuracy: 0.1379 - 7s/epoch - 123ms/step Epoch 2/50 61/61 - 6s - loss: 2.4147 - accuracy: 0.1300 - val_loss: 2.4252 - val_accuracy: 0.1285 - 6s/epoch - 105ms/step Epoch 3/50 61/61 - 6s - loss: 2.4214 - accuracy: 0.1398 - val_loss: 2.4081 - val_accuracy: 0.1379 - 6s/epoch - 100ms/step Epoch 4/50 61/61 - 6s - loss: 2.3789 - accuracy: 0.1521 - val_loss: 2.3051 - val_accuracy: 0.1612 - 6s/epoch - 104ms/step Epoch 5/50 61/61 - 6s - loss: 2.1494 - accuracy: 0.2313 - val_loss: 1.9198 - val_accuracy: 0.3551 - 6s/epoch - 106ms/step Epoch 6/50 61/61 - 6s - loss: 1.7694 - accuracy: 0.3543 - val_loss: 1.5920 - val_accuracy: 0.4229 - 6s/epoch - 103ms/step Epoch 7/50 61/61 - 6s - loss: 1.5474 - accuracy: 0.4474 - val_loss: 1.4632 - val_accuracy: 0.5117 - 6s/epoch - 101ms/step Epoch 8/50 61/61 - 7s - loss: 1.3990 - accuracy: 0.5069 - val_loss: 1.1936 - val_accuracy: 0.5888 - 7s/epoch - 107ms/step Epoch 9/50 61/61 - 7s - loss: 1.2051 - accuracy: 0.5688 - val_loss: 1.4414 - val_accuracy: 0.5070 - 7s/epoch - 107ms/step Epoch 10/50 61/61 - 6s - loss: 1.1873 - accuracy: 0.5763 - val_loss: 1.1966 - val_accuracy: 0.5794 - 6s/epoch - 103ms/step Epoch 11/50 61/61 - 6s - loss: 1.0036 - accuracy: 0.6428 - val_loss: 1.0713 - val_accuracy: 0.6495 - 6s/epoch - 104ms/step Epoch 12/50 61/61 - 6s - loss: 0.9437 - accuracy: 0.6688 - val_loss: 0.9207 - val_accuracy: 0.6729 - 6s/epoch - 98ms/step Epoch 13/50 61/61 - 7s - loss: 0.8662 - accuracy: 0.6855 - val_loss: 0.8806 - val_accuracy: 0.7126 - 7s/epoch - 110ms/step Epoch 14/50 61/61 - 7s - loss: 0.8069 - accuracy: 0.7120 - val_loss: 0.8839 - val_accuracy: 0.7173 - 7s/epoch - 110ms/step Epoch 15/50 61/61 - 7s - loss: 0.7473 - accuracy: 0.7356 - val_loss: 0.8143 - val_accuracy: 0.7430 - 7s/epoch - 107ms/step Epoch 16/50 61/61 - 6s - loss: 0.7139 - accuracy: 0.7466 - val_loss: 0.8805 - val_accuracy: 0.7290 - 6s/epoch - 106ms/step Epoch 17/50 61/61 - 7s - loss: 0.6794 - accuracy: 0.7533 - val_loss: 0.7486 - val_accuracy: 0.7453 - 7s/epoch - 107ms/step Epoch 18/50 61/61 - 7s - loss: 0.6679 - accuracy: 0.7632 - val_loss: 0.7543 - val_accuracy: 0.7523 - 7s/epoch - 107ms/step Epoch 19/50 61/61 - 6s - loss: 0.5526 - accuracy: 0.7957 - val_loss: 0.7203 - val_accuracy: 0.7757 - 6s/epoch - 104ms/step Epoch 20/50 61/61 - 6s - loss: 0.5371 - accuracy: 0.8009 - val_loss: 0.9041 - val_accuracy: 0.7360 - 6s/epoch - 104ms/step Epoch 21/50 61/61 - 6s - loss: 0.5016 - accuracy: 0.8227 - val_loss: 0.6814 - val_accuracy: 0.8061 - 6s/epoch - 104ms/step Epoch 22/50 61/61 - 6s - loss: 0.4542 - accuracy: 0.8329 - val_loss: 0.6464 - val_accuracy: 0.8061 - 6s/epoch - 99ms/step Epoch 23/50 61/61 - 6s - loss: 0.4387 - accuracy: 0.8422 - val_loss: 0.6651 - val_accuracy: 0.7921 - 6s/epoch - 102ms/step Epoch 24/50 61/61 - 6s - loss: 0.4250 - accuracy: 0.8472 - val_loss: 0.7764 - val_accuracy: 0.7710 - 6s/epoch - 103ms/step Epoch 25/50 61/61 - 7s - loss: 0.3783 - accuracy: 0.8583 - val_loss: 0.6588 - val_accuracy: 0.8154 - 7s/epoch - 109ms/step Epoch 26/50 61/61 - 6s - loss: 0.3841 - accuracy: 0.8555 - val_loss: 0.6227 - val_accuracy: 0.8201 - 6s/epoch - 103ms/step Epoch 27/50 61/61 - 6s - loss: 0.3339 - accuracy: 0.8700 - val_loss: 0.6962 - val_accuracy: 0.7944 - 6s/epoch - 100ms/step Epoch 28/50 61/61 - 6s - loss: 0.3126 - accuracy: 0.8830 - val_loss: 0.6416 - val_accuracy: 0.8318 - 6s/epoch - 102ms/step Epoch 29/50 61/61 - 6s - loss: 0.3219 - accuracy: 0.8822 - val_loss: 0.7842 - val_accuracy: 0.7897 - 6s/epoch - 97ms/step Epoch 30/50 61/61 - 6s - loss: 0.3281 - accuracy: 0.8799 - val_loss: 0.6454 - val_accuracy: 0.8248 - 6s/epoch - 101ms/step Epoch 31/50 61/61 - 6s - loss: 0.3063 - accuracy: 0.8843 - val_loss: 0.7224 - val_accuracy: 0.7921 - 6s/epoch - 97ms/step Epoch 32/50 61/61 - 7s - loss: 0.2605 - accuracy: 0.9015 - val_loss: 0.6440 - val_accuracy: 0.8388 - 7s/epoch - 109ms/step Epoch 33/50 61/61 - 7s - loss: 0.2490 - accuracy: 0.9015 - val_loss: 0.7124 - val_accuracy: 0.8201 - 7s/epoch - 111ms/step Epoch 34/50 61/61 - 6s - loss: 0.2567 - accuracy: 0.9012 - val_loss: 0.6405 - val_accuracy: 0.8131 - 6s/epoch - 105ms/step Epoch 35/50 61/61 - 6s - loss: 0.2626 - accuracy: 0.8997 - val_loss: 0.8338 - val_accuracy: 0.7921 - 6s/epoch - 106ms/step Epoch 36/50 61/61 - 6s - loss: 0.2509 - accuracy: 0.9043 - val_loss: 0.6245 - val_accuracy: 0.8458 - 6s/epoch - 105ms/step Epoch 37/50 61/61 - 6s - loss: 0.2209 - accuracy: 0.9186 - val_loss: 0.6883 - val_accuracy: 0.8201 - 6s/epoch - 98ms/step Epoch 38/50 61/61 - 6s - loss: 0.2431 - accuracy: 0.9093 - val_loss: 0.6749 - val_accuracy: 0.8131 - 6s/epoch - 101ms/step Epoch 39/50 61/61 - 6s - loss: 0.2325 - accuracy: 0.9124 - val_loss: 0.7944 - val_accuracy: 0.7921 - 6s/epoch - 99ms/step Epoch 40/50 61/61 - 6s - loss: 0.2552 - accuracy: 0.9038 - val_loss: 0.6227 - val_accuracy: 0.8271 - 6s/epoch - 97ms/step Epoch 41/50 61/61 - 6s - loss: 0.1817 - accuracy: 0.9332 - val_loss: 0.6876 - val_accuracy: 0.8178 - 6s/epoch - 98ms/step Epoch 42/50 61/61 - 6s - loss: 0.2605 - accuracy: 0.9028 - val_loss: 0.6727 - val_accuracy: 0.8271 - 6s/epoch - 104ms/step Epoch 43/50 61/61 - 6s - loss: 0.1990 - accuracy: 0.9254 - val_loss: 0.6308 - val_accuracy: 0.8364 - 6s/epoch - 105ms/step Epoch 44/50 61/61 - 6s - loss: 0.1914 - accuracy: 0.9322 - val_loss: 0.5838 - val_accuracy: 0.8645 - 6s/epoch - 97ms/step Epoch 45/50 61/61 - 6s - loss: 0.1593 - accuracy: 0.9410 - val_loss: 0.6850 - val_accuracy: 0.8318 - 6s/epoch - 100ms/step Epoch 46/50 61/61 - 6s - loss: 0.1828 - accuracy: 0.9327 - val_loss: 0.6906 - val_accuracy: 0.8364 - 6s/epoch - 105ms/step Epoch 47/50 61/61 - 6s - loss: 0.1739 - accuracy: 0.9355 - val_loss: 0.8667 - val_accuracy: 0.7921 - 6s/epoch - 100ms/step Epoch 48/50 61/61 - 6s - loss: 0.1701 - accuracy: 0.9374 - val_loss: 0.5989 - val_accuracy: 0.8458 - 6s/epoch - 104ms/step Epoch 49/50 61/61 - 7s - loss: 0.1656 - accuracy: 0.9410 - val_loss: 0.7510 - val_accuracy: 0.8131 - 7s/epoch - 110ms/step Epoch 50/50 61/61 - 6s - loss: 0.1375 - accuracy: 0.9491 - val_loss: 0.6773 - val_accuracy: 0.8598 - 6s/epoch - 104ms/step
Model Evaluation
plt.plot(complex_adam_opt_history.history['accuracy'])
plt.plot(complex_adam_opt_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- This model behaves much like the non adjusted adam optimizer model.
- It does not seem to improve much past epoch 25.
Loss
plt.plot(complex_adam_opt_history.history['loss'])
plt.plot(complex_adam_opt_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- Again similar performance as the non adjusted adam model.
- It does not seem to improve much past epoch 35.
Evaluate the model on test data
complex_history_adam_opt_accuracy = complex_model_adam_opt.evaluate(X_test_normalized,
y_test_encoded,
#verbose=2
)
15/15 [==============================] - 1s 41ms/step - loss: 0.6181 - accuracy: 0.8400
best_model_accuracy = complex_adam_opt_history.history['accuracy'][np.argmin(complex_adam_opt_history.history['loss'])]
best_model_accuracy
0.9490512013435364
#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = complex_history_adam_opt_accuracy[1]
print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])
accuracy 0.949051 Name: Complex Model with Adam optimizer adjusted, dtype: float64 accuracy 0.84 Name: Complex Model with Adam optimizer adjusted, dtype: float64
Observations¶
- Test Accuracy is 84%
- Training model accuracy with least loss is 95% so a bit of overfitting there.
Plotting the Confusion Matrix
# Here we would get the output as probablities for each category
y_pred=complex_model_adam_opt.predict(X_test_normalized)
15/15 [==============================] - 1s 38ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Observations¶
- Common Chickweed, Sheperds Purse, and Common Wheat did not improve from base complex model.
- Fat Hen has improved a bit from 8 misidentified to 6.
- Loose Silky-bent and Maize have been 100% identified correctly.
- Scentless Mayweed returned to 11 misidentified from 12.
- Black-grass(14) remains the largest misclassification.
Model adjusted by reducing the Learning Rate:¶
In this model we will use the ReduceLRonPlateau to adjust the model. We will also add Batch Normalization.
ReduceLRonPlateau() is a function that will be used to decrease the learning rate by some factor, if the loss is not decreasing for some time. This may start decreasing the loss at a smaller learning rate. There is a possibility that the loss may still not decrease. This may lead to executing the learning rate reduction again in an attempt to achieve a lower loss.
# Code to monitor val_accuracy
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy',
patience=2, #was 3
verbose=1,
factor=0.5,
min_lr=0.00001)
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
model_name = "Complex Model with learning rate reduction and batch normalization"
h, w, c = X_train_normalized[1000].shape
print('width: ', w)
print('height: ', h)
print('channel:', c)
# Intializing a sequential model
lron_base_model = Sequential()
# create convolution and max-pooling layers with input size of images
lron_base_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h,w,c)))
lron_base_model.add(BatchNormalization())
lron_base_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_base_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
lron_base_model.add(BatchNormalization())
lron_base_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_base_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_base_model.add(BatchNormalization())
lron_base_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_base_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_base_model.add(BatchNormalization())
lron_base_model.add(MaxPooling2D((2, 2), padding = 'same'))
# flatten the output of the conv layer
lron_base_model.add(Flatten())
# add a fully connected layer
lron_base_model.add(Dense(128, activation='relu'))
lron_base_model.add(Dropout(0.3))
# add a fully connected layer
lron_base_model.add(Dense(512, activation='relu'))
lron_base_model.add(Dropout(0.3))
# add a fully connected layer
lron_base_model.add(Dense(256, activation='relu'))
lron_base_model.add(Dropout(0.3))
# output layer with 12 neurons and softmax
lron_base_model.add(Dense(12, activation='softmax'))
# Adam Optimizer
opt=Adam()
# Compile the model
lron_base_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# generate the summary
lron_base_model.summary()
width: 64
height: 64
channel: 3
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 32) 896
batch_normalization (Batch (None, 64, 64, 32) 128
Normalization)
max_pooling2d (MaxPooling2 (None, 32, 32, 32) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 64) 18496
batch_normalization_1 (Bat (None, 32, 32, 64) 256
chNormalization)
max_pooling2d_1 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_2 (Conv2D) (None, 16, 16, 128) 73856
batch_normalization_2 (Bat (None, 16, 16, 128) 512
chNormalization)
max_pooling2d_2 (MaxPoolin (None, 8, 8, 128) 0
g2D)
conv2d_3 (Conv2D) (None, 8, 8, 128) 147584
batch_normalization_3 (Bat (None, 8, 8, 128) 512
chNormalization)
max_pooling2d_3 (MaxPoolin (None, 4, 4, 128) 0
g2D)
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 128) 262272
dropout (Dropout) (None, 128) 0
dense_1 (Dense) (None, 512) 66048
dropout_1 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 256) 131328
dropout_2 (Dropout) (None, 256) 0
dense_3 (Dense) (None, 12) 3084
=================================================================
Total params: 704972 (2.69 MB)
Trainable params: 704268 (2.69 MB)
Non-trainable params: 704 (2.75 KB)
_________________________________________________________________
Fitting the model on the train data
lron_base_history = lron_base_model.fit(
X_train_normalized, y_train_encoded,
epochs=epochs,
validation_data=(X_val_normalized,y_val_encoded),
batch_size=batch_size,
verbose=2,callbacks=[learning_rate_reduction])
Epoch 1/25 121/121 - 19s - loss: 2.0012 - accuracy: 0.3312 - val_loss: 7.4617 - val_accuracy: 0.1379 - lr: 0.0010 - 19s/epoch - 156ms/step Epoch 2/25 121/121 - 17s - loss: 1.3346 - accuracy: 0.5545 - val_loss: 13.4252 - val_accuracy: 0.1379 - lr: 0.0010 - 17s/epoch - 139ms/step Epoch 3/25 121/121 - 16s - loss: 1.0517 - accuracy: 0.6460 - val_loss: 4.0732 - val_accuracy: 0.1869 - lr: 0.0010 - 16s/epoch - 136ms/step Epoch 4/25 121/121 - 17s - loss: 0.8448 - accuracy: 0.7174 - val_loss: 13.2370 - val_accuracy: 0.1379 - lr: 0.0010 - 17s/epoch - 138ms/step Epoch 5/25 121/121 - 17s - loss: 0.7431 - accuracy: 0.7515 - val_loss: 3.0628 - val_accuracy: 0.3925 - lr: 0.0010 - 17s/epoch - 139ms/step Epoch 6/25 121/121 - 16s - loss: 0.6106 - accuracy: 0.7915 - val_loss: 1.8700 - val_accuracy: 0.4883 - lr: 0.0010 - 16s/epoch - 133ms/step Epoch 7/25 121/121 - 15s - loss: 0.5493 - accuracy: 0.8121 - val_loss: 0.8549 - val_accuracy: 0.7687 - lr: 0.0010 - 15s/epoch - 128ms/step Epoch 8/25 121/121 - 16s - loss: 0.4791 - accuracy: 0.8352 - val_loss: 1.2403 - val_accuracy: 0.5981 - lr: 0.0010 - 16s/epoch - 134ms/step Epoch 9/25 121/121 - 18s - loss: 0.4797 - accuracy: 0.8310 - val_loss: 0.6532 - val_accuracy: 0.8107 - lr: 0.0010 - 18s/epoch - 145ms/step Epoch 10/25 121/121 - 18s - loss: 0.3922 - accuracy: 0.8638 - val_loss: 1.3370 - val_accuracy: 0.5888 - lr: 0.0010 - 18s/epoch - 149ms/step Epoch 11/25 Epoch 11: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. 121/121 - 18s - loss: 0.3685 - accuracy: 0.8695 - val_loss: 0.7262 - val_accuracy: 0.7850 - lr: 0.0010 - 18s/epoch - 152ms/step Epoch 12/25 121/121 - 19s - loss: 0.2520 - accuracy: 0.9043 - val_loss: 0.5732 - val_accuracy: 0.8435 - lr: 5.0000e-04 - 19s/epoch - 156ms/step Epoch 13/25 121/121 - 18s - loss: 0.1979 - accuracy: 0.9264 - val_loss: 0.7829 - val_accuracy: 0.8201 - lr: 5.0000e-04 - 18s/epoch - 150ms/step Epoch 14/25 121/121 - 18s - loss: 0.1824 - accuracy: 0.9324 - val_loss: 0.5219 - val_accuracy: 0.8621 - lr: 5.0000e-04 - 18s/epoch - 145ms/step Epoch 15/25 121/121 - 18s - loss: 0.1678 - accuracy: 0.9389 - val_loss: 1.3576 - val_accuracy: 0.7033 - lr: 5.0000e-04 - 18s/epoch - 149ms/step Epoch 16/25 Epoch 16: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. 121/121 - 17s - loss: 0.1665 - accuracy: 0.9389 - val_loss: 1.1790 - val_accuracy: 0.7266 - lr: 5.0000e-04 - 17s/epoch - 143ms/step Epoch 17/25 121/121 - 18s - loss: 0.1293 - accuracy: 0.9530 - val_loss: 0.4625 - val_accuracy: 0.8949 - lr: 2.5000e-04 - 18s/epoch - 146ms/step Epoch 18/25 121/121 - 17s - loss: 0.1205 - accuracy: 0.9605 - val_loss: 0.6072 - val_accuracy: 0.8668 - lr: 2.5000e-04 - 17s/epoch - 143ms/step Epoch 19/25 Epoch 19: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. 121/121 - 18s - loss: 0.0898 - accuracy: 0.9662 - val_loss: 0.5988 - val_accuracy: 0.8598 - lr: 2.5000e-04 - 18s/epoch - 148ms/step Epoch 20/25 121/121 - 17s - loss: 0.0773 - accuracy: 0.9748 - val_loss: 0.4249 - val_accuracy: 0.9182 - lr: 1.2500e-04 - 17s/epoch - 138ms/step Epoch 21/25 121/121 - 16s - loss: 0.0630 - accuracy: 0.9797 - val_loss: 0.4789 - val_accuracy: 0.8925 - lr: 1.2500e-04 - 16s/epoch - 131ms/step Epoch 22/25 Epoch 22: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05. 121/121 - 16s - loss: 0.0716 - accuracy: 0.9743 - val_loss: 0.5871 - val_accuracy: 0.8949 - lr: 1.2500e-04 - 16s/epoch - 131ms/step Epoch 23/25 121/121 - 17s - loss: 0.0584 - accuracy: 0.9800 - val_loss: 0.5589 - val_accuracy: 0.9019 - lr: 6.2500e-05 - 17s/epoch - 143ms/step Epoch 24/25 Epoch 24: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05. 121/121 - 17s - loss: 0.0498 - accuracy: 0.9831 - val_loss: 0.4390 - val_accuracy: 0.9182 - lr: 6.2500e-05 - 17s/epoch - 140ms/step Epoch 25/25 121/121 - 18s - loss: 0.0446 - accuracy: 0.9857 - val_loss: 0.4321 - val_accuracy: 0.9206 - lr: 3.1250e-05 - 18s/epoch - 149ms/step
Model Evaluation
plt.plot(lron_base_history.history['accuracy'])
plt.plot(lron_base_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- Fom the graph we can see that the training accuracy of the model was reaching overfitting.
- The validation accuracy was oscillating and divergent.
plt.plot(lron_base_history.history['loss'])
plt.plot(lron_base_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations¶
- Loss between Training and validation remains pretty stable after epoch 13.
Evaluate the model on test data
lron_base_model_test_accuracy = lron_base_model.evaluate(X_test_normalized,
y_test_encoded,
#verbose=2
)
15/15 [==============================] - 1s 62ms/step - loss: 0.4490 - accuracy: 0.9116
best_model_accuracy = lron_base_history.history['accuracy'][np.argmin(lron_base_history.history['loss'])]
best_model_accuracy
0.98570317029953
#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = lron_base_model_test_accuracy[1]
print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])
accuracy 0.985703 Name: Complex Model with learning rate reduction and batch normalization, dtype: float64 accuracy 0.911579 Name: Complex Model with learning rate reduction and batch normalization, dtype: float64
Observations¶
- Test Accuracy is 91%
- Training model accuracy with least loss is 99%.
Plotting the Confusion Matrix
# Complete the code to obtain the output probabilities
y_pred=lron_base_model.predict(X_test_normalized)
15/15 [==============================] - 1s 61ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg) # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Observations¶
- Black-grass has improved from 14 to 12 misindentified
- Common Chickweed, Common Wheat, Fat Hen, Scentless Mayweed have all improved.
- Maize has been 100% identified correctly.
- Sheperds Purse has increased in misidentification.
- Loose Silky-bent is still highly misclassified.
Model with basic Data Augmentation¶
In this model we will add basic data augmentation techniques and keep the batch normalization and learning rate reduction.
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# Complete the code to set the rotation_range to 20
train_datagen = ImageDataGenerator(
rotation_range=20,
fill_mode='nearest'
)
model_name = "Complex Model with batch normalization, learning rate reduction and basic data augmentation"
h, w, c = X_train_normalized[1000].shape
print('width: ', w)
print('height: ', h)
print('channel:', c)
# Intializing a sequential model
lron_da_model = Sequential()
# create convolution and max-pooling layers with input size of images
lron_da_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h,w,c)))
lron_da_model.add(BatchNormalization())
lron_da_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_da_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
lron_da_model.add(BatchNormalization())
lron_da_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_da_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da_model.add(BatchNormalization())
lron_da_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_da_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da_model.add(BatchNormalization())
lron_da_model.add(MaxPooling2D((2, 2), padding = 'same'))
# flatten the output of the conv layer
lron_da_model.add(Flatten())
# add a fully connected layer
lron_da_model.add(Dense(128, activation='relu'))
lron_da_model.add(Dropout(0.3))
# add a fully connected layer
lron_da_model.add(Dense(512, activation='relu'))
lron_da_model.add(Dropout(0.3))
# add a fully connected layer
lron_da_model.add(Dense(256, activation='relu'))
lron_da_model.add(Dropout(0.3))
# output layer with 12 neurons and softmax
lron_da_model.add(Dense(12, activation='softmax'))
# Adam Optimizer
opt=Adam()
# Compile the model
lron_da_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# generate the summary
lron_da_model.summary()
width: 64
height: 64
channel: 3
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 32) 896
batch_normalization (Batch (None, 64, 64, 32) 128
Normalization)
max_pooling2d (MaxPooling2 (None, 32, 32, 32) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 64) 18496
batch_normalization_1 (Bat (None, 32, 32, 64) 256
chNormalization)
max_pooling2d_1 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_2 (Conv2D) (None, 16, 16, 128) 73856
batch_normalization_2 (Bat (None, 16, 16, 128) 512
chNormalization)
max_pooling2d_2 (MaxPoolin (None, 8, 8, 128) 0
g2D)
conv2d_3 (Conv2D) (None, 8, 8, 128) 147584
batch_normalization_3 (Bat (None, 8, 8, 128) 512
chNormalization)
max_pooling2d_3 (MaxPoolin (None, 4, 4, 128) 0
g2D)
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 128) 262272
dropout (Dropout) (None, 128) 0
dense_1 (Dense) (None, 512) 66048
dropout_1 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 256) 131328
dropout_2 (Dropout) (None, 256) 0
dense_3 (Dense) (None, 12) 3084
=================================================================
Total params: 704972 (2.69 MB)
Trainable params: 704268 (2.69 MB)
Non-trainable params: 704 (2.75 KB)
_________________________________________________________________
Fitting the model on the train data
lron_da_history = lron_da_model.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=32,
shuffle=False),
epochs=epochs,
validation_data=(X_val_normalized,y_val_encoded),
verbose=1,
callbacks=[learning_rate_reduction])
Epoch 1/25 121/121 [==============================] - 19s 146ms/step - loss: 2.0073 - accuracy: 0.3174 - val_loss: 8.4350 - val_accuracy: 0.1379 - lr: 0.0010 Epoch 2/25 121/121 [==============================] - 19s 154ms/step - loss: 1.3775 - accuracy: 0.5217 - val_loss: 15.4300 - val_accuracy: 0.1379 - lr: 0.0010 Epoch 3/25 121/121 [==============================] - ETA: 0s - loss: 1.1028 - accuracy: 0.6291 Epoch 3: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. 121/121 [==============================] - 19s 154ms/step - loss: 1.1028 - accuracy: 0.6291 - val_loss: 15.1994 - val_accuracy: 0.1379 - lr: 0.0010 Epoch 4/25 121/121 [==============================] - 18s 152ms/step - loss: 0.8084 - accuracy: 0.7255 - val_loss: 10.1189 - val_accuracy: 0.1379 - lr: 5.0000e-04 Epoch 5/25 121/121 [==============================] - 19s 153ms/step - loss: 0.6915 - accuracy: 0.7635 - val_loss: 2.3219 - val_accuracy: 0.4463 - lr: 5.0000e-04 Epoch 6/25 121/121 [==============================] - 18s 151ms/step - loss: 0.5962 - accuracy: 0.7926 - val_loss: 1.1982 - val_accuracy: 0.6636 - lr: 5.0000e-04 Epoch 7/25 121/121 [==============================] - 19s 157ms/step - loss: 0.5363 - accuracy: 0.8097 - val_loss: 0.8725 - val_accuracy: 0.7500 - lr: 5.0000e-04 Epoch 8/25 121/121 [==============================] - 17s 138ms/step - loss: 0.5366 - accuracy: 0.8134 - val_loss: 1.0062 - val_accuracy: 0.6939 - lr: 5.0000e-04 Epoch 9/25 121/121 [==============================] - 18s 145ms/step - loss: 0.4544 - accuracy: 0.8378 - val_loss: 0.7697 - val_accuracy: 0.7687 - lr: 5.0000e-04 Epoch 10/25 121/121 [==============================] - 18s 152ms/step - loss: 0.3830 - accuracy: 0.8550 - val_loss: 1.4741 - val_accuracy: 0.5678 - lr: 5.0000e-04 Epoch 11/25 121/121 [==============================] - 20s 165ms/step - loss: 0.3838 - accuracy: 0.8653 - val_loss: 0.5951 - val_accuracy: 0.8528 - lr: 5.0000e-04 Epoch 12/25 121/121 [==============================] - 20s 169ms/step - loss: 0.3689 - accuracy: 0.8718 - val_loss: 0.6033 - val_accuracy: 0.8248 - lr: 5.0000e-04 Epoch 13/25 121/121 [==============================] - 19s 158ms/step - loss: 0.3250 - accuracy: 0.8802 - val_loss: 0.5281 - val_accuracy: 0.8715 - lr: 5.0000e-04 Epoch 14/25 121/121 [==============================] - 19s 158ms/step - loss: 0.2926 - accuracy: 0.8913 - val_loss: 0.6886 - val_accuracy: 0.8084 - lr: 5.0000e-04 Epoch 15/25 121/121 [==============================] - ETA: 0s - loss: 0.3045 - accuracy: 0.8885 Epoch 15: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. 121/121 [==============================] - 19s 155ms/step - loss: 0.3045 - accuracy: 0.8885 - val_loss: 0.6094 - val_accuracy: 0.8201 - lr: 5.0000e-04 Epoch 16/25 121/121 [==============================] - 18s 152ms/step - loss: 0.2171 - accuracy: 0.9189 - val_loss: 0.5359 - val_accuracy: 0.8505 - lr: 2.5000e-04 Epoch 17/25 121/121 [==============================] - ETA: 0s - loss: 0.1898 - accuracy: 0.9233 Epoch 17: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. 121/121 [==============================] - 18s 152ms/step - loss: 0.1898 - accuracy: 0.9233 - val_loss: 0.7065 - val_accuracy: 0.8178 - lr: 2.5000e-04 Epoch 18/25 121/121 [==============================] - 19s 154ms/step - loss: 0.1758 - accuracy: 0.9361 - val_loss: 0.4001 - val_accuracy: 0.9042 - lr: 1.2500e-04 Epoch 19/25 121/121 [==============================] - 19s 157ms/step - loss: 0.1443 - accuracy: 0.9444 - val_loss: 0.4023 - val_accuracy: 0.9042 - lr: 1.2500e-04 Epoch 20/25 121/121 [==============================] - 19s 158ms/step - loss: 0.1386 - accuracy: 0.9488 - val_loss: 0.4146 - val_accuracy: 0.9112 - lr: 1.2500e-04 Epoch 21/25 121/121 [==============================] - 19s 161ms/step - loss: 0.1340 - accuracy: 0.9488 - val_loss: 0.4319 - val_accuracy: 0.9042 - lr: 1.2500e-04 Epoch 22/25 121/121 [==============================] - ETA: 0s - loss: 0.1251 - accuracy: 0.9509 Epoch 22: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05. 121/121 [==============================] - 19s 156ms/step - loss: 0.1251 - accuracy: 0.9509 - val_loss: 0.4738 - val_accuracy: 0.8925 - lr: 1.2500e-04 Epoch 23/25 121/121 [==============================] - 19s 157ms/step - loss: 0.1145 - accuracy: 0.9561 - val_loss: 0.4030 - val_accuracy: 0.9042 - lr: 6.2500e-05 Epoch 24/25 121/121 [==============================] - ETA: 0s - loss: 0.1121 - accuracy: 0.9587 Epoch 24: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05. 121/121 [==============================] - 18s 151ms/step - loss: 0.1121 - accuracy: 0.9587 - val_loss: 0.4251 - val_accuracy: 0.9065 - lr: 6.2500e-05 Epoch 25/25 121/121 [==============================] - 19s 156ms/step - loss: 0.0963 - accuracy: 0.9665 - val_loss: 0.4347 - val_accuracy: 0.9089 - lr: 3.1250e-05
Model Evaluation
plt.plot(lron_da_history.history['accuracy'])
plt.plot(lron_da_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations¶
- We still see signs of the training data overfitting and oscillations of the validation data until around epoch 16.
- Training and validation are still pretty divergent.
plt.plot(lron_da_history.history['loss'])
plt.plot(lron_da_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- Loss flattens out beyond epoch 10.
Evaluate the model on test data
lron_da_test_accuracy = lron_da_model.evaluate(X_test_normalized,
y_test_encoded,
#verbose=2
)
15/15 [==============================] - 1s 66ms/step - loss: 0.4274 - accuracy: 0.8947
best_model_accuracy = lron_da_history.history['accuracy'][np.argmin(lron_da_history.history['loss'])]
best_model_accuracy
0.9664673805236816
#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = lron_da_test_accuracy[1]
print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])
accuracy 0.966467 Name: Complex Model with batch normalization, learning rate reduction and basic data augmentation, dtype: float64 accuracy 0.894737 Name: Complex Model with batch normalization, learning rate reduction and basic data augmentation, dtype: float64
Observations¶
- Test Accuracy is 89%, so a slight improvement
- Model accuracy with least loss is 97%, so overfitting a bit less.
Plotting the Confusion Matrix
# Complete the code to obtain the output probabilities
y_pred=lron_da_model.predict(X_test_normalized)
15/15 [==============================] - 1s 63ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg) # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Observations¶
- Black-grass has improved from 8 to 3 misindentified
- Common Chickweed, Common Wheat, Fat Hen, Scentless Mayweed have all remained a low percentage of misidentifications.
- Maize has been 100% identified correctly.
- Sheperds Purse has increased by 1 in misidentification.
- Loose Silky-bent is now the largest misclassification.
- Most others remain about the same level of misclassification.
Model adjustment by adding steps per epoch¶
We will add steps per epoch to set how many times we get a new batch from the generator to try and improve the model. This will end up discarding 7 images in training. This will work in conjunction with ReduceLROnPlateau to better decrease the learning rate.
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# Complete the code to set the rotation_range to 20
train_datagen = ImageDataGenerator(
rotation_range=20,
fill_mode='nearest'
)
model_name = "Complex Model with batch normalization, learning rate reduction, basic data augmentation and steps per epoch"
h, w, c = X_train_normalized[1000].shape
print('width: ', w)
print('height: ', h)
print('channel:', c)
# Intializing a sequential model
lron_da_spe_model = Sequential()
# create convolution and max-pooling layers with input size of images
lron_da_spe_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h,w,c)))
lron_da_spe_model.add(BatchNormalization())
lron_da_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_da_spe_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
lron_da_spe_model.add(BatchNormalization())
lron_da_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_da_spe_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da_spe_model.add(BatchNormalization())
lron_da_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_da_spe_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da_spe_model.add(BatchNormalization())
lron_da_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))
# flatten the output of the conv layer
lron_da_spe_model.add(Flatten())
# add a fully connected layer
lron_da_spe_model.add(Dense(128, activation='relu'))
lron_da_spe_model.add(Dropout(0.3))
# add a fully connected layer
lron_da_spe_model.add(Dense(512, activation='relu'))
lron_da_spe_model.add(Dropout(0.3))
# add a fully connected layer
lron_da_spe_model.add(Dense(256, activation='relu'))
lron_da_spe_model.add(Dropout(0.3))
# output layer with 12 neurons and softmax
lron_da_spe_model.add(Dense(12, activation='softmax'))
# Adam Optimizer
opt=Adam()
# Compile the model
lron_da_spe_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# generate the summary
lron_da_spe_model.summary()
width: 64
height: 64
channel: 3
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 32) 896
batch_normalization (Batch (None, 64, 64, 32) 128
Normalization)
max_pooling2d (MaxPooling2 (None, 32, 32, 32) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 64) 18496
batch_normalization_1 (Bat (None, 32, 32, 64) 256
chNormalization)
max_pooling2d_1 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_2 (Conv2D) (None, 16, 16, 128) 73856
batch_normalization_2 (Bat (None, 16, 16, 128) 512
chNormalization)
max_pooling2d_2 (MaxPoolin (None, 8, 8, 128) 0
g2D)
conv2d_3 (Conv2D) (None, 8, 8, 128) 147584
batch_normalization_3 (Bat (None, 8, 8, 128) 512
chNormalization)
max_pooling2d_3 (MaxPoolin (None, 4, 4, 128) 0
g2D)
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 128) 262272
dropout (Dropout) (None, 128) 0
dense_1 (Dense) (None, 512) 66048
dropout_1 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 256) 131328
dropout_2 (Dropout) (None, 256) 0
dense_3 (Dense) (None, 12) 3084
=================================================================
Total params: 704972 (2.69 MB)
Trainable params: 704268 (2.69 MB)
Non-trainable params: 704 (2.75 KB)
_________________________________________________________________
Fitting the model on the train data
lron_da_spe_history = lron_da_spe_model.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size, #errors out of range
validation_data=(X_val_normalized,y_val_encoded),
verbose=1,callbacks=[learning_rate_reduction])
Epoch 1/25 120/120 [==============================] - 20s 151ms/step - loss: 2.0092 - accuracy: 0.3166 - val_loss: 8.3740 - val_accuracy: 0.1379 - lr: 0.0010 Epoch 2/25 120/120 [==============================] - 19s 155ms/step - loss: 1.3914 - accuracy: 0.5266 - val_loss: 11.3857 - val_accuracy: 0.1379 - lr: 0.0010 Epoch 3/25 120/120 [==============================] - ETA: 0s - loss: 1.1237 - accuracy: 0.6275 Epoch 3: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. 120/120 [==============================] - 19s 157ms/step - loss: 1.1237 - accuracy: 0.6275 - val_loss: 15.8686 - val_accuracy: 0.1379 - lr: 0.0010 Epoch 4/25 120/120 [==============================] - 19s 162ms/step - loss: 0.8396 - accuracy: 0.7132 - val_loss: 6.4671 - val_accuracy: 0.1565 - lr: 5.0000e-04 Epoch 5/25 120/120 [==============================] - 19s 162ms/step - loss: 0.6952 - accuracy: 0.7507 - val_loss: 2.5973 - val_accuracy: 0.3879 - lr: 5.0000e-04 Epoch 6/25 120/120 [==============================] - 19s 159ms/step - loss: 0.6119 - accuracy: 0.7832 - val_loss: 0.7144 - val_accuracy: 0.7710 - lr: 5.0000e-04 Epoch 7/25 120/120 [==============================] - 19s 157ms/step - loss: 0.5156 - accuracy: 0.8144 - val_loss: 0.5888 - val_accuracy: 0.8248 - lr: 5.0000e-04 Epoch 8/25 120/120 [==============================] - 18s 150ms/step - loss: 0.4656 - accuracy: 0.8391 - val_loss: 0.7592 - val_accuracy: 0.7850 - lr: 5.0000e-04 Epoch 9/25 120/120 [==============================] - ETA: 0s - loss: 0.4195 - accuracy: 0.8474 Epoch 9: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. 120/120 [==============================] - 18s 149ms/step - loss: 0.4195 - accuracy: 0.8474 - val_loss: 0.9602 - val_accuracy: 0.7243 - lr: 5.0000e-04 Epoch 10/25 120/120 [==============================] - 18s 153ms/step - loss: 0.3632 - accuracy: 0.8708 - val_loss: 0.4819 - val_accuracy: 0.8621 - lr: 2.5000e-04 Epoch 11/25 120/120 [==============================] - 18s 152ms/step - loss: 0.2995 - accuracy: 0.8852 - val_loss: 0.4726 - val_accuracy: 0.8528 - lr: 2.5000e-04 Epoch 12/25 120/120 [==============================] - ETA: 0s - loss: 0.2853 - accuracy: 0.8954 Epoch 12: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. 120/120 [==============================] - 19s 154ms/step - loss: 0.2853 - accuracy: 0.8954 - val_loss: 0.4542 - val_accuracy: 0.8598 - lr: 2.5000e-04 Epoch 13/25 120/120 [==============================] - 19s 159ms/step - loss: 0.2356 - accuracy: 0.9119 - val_loss: 0.4342 - val_accuracy: 0.8785 - lr: 1.2500e-04 Epoch 14/25 120/120 [==============================] - 19s 157ms/step - loss: 0.2166 - accuracy: 0.9130 - val_loss: 0.4105 - val_accuracy: 0.8925 - lr: 1.2500e-04 Epoch 15/25 120/120 [==============================] - 18s 150ms/step - loss: 0.1997 - accuracy: 0.9263 - val_loss: 0.4178 - val_accuracy: 0.8902 - lr: 1.2500e-04 Epoch 16/25 120/120 [==============================] - ETA: 0s - loss: 0.1965 - accuracy: 0.9266 Epoch 16: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05. 120/120 [==============================] - 18s 153ms/step - loss: 0.1965 - accuracy: 0.9266 - val_loss: 0.4869 - val_accuracy: 0.8879 - lr: 1.2500e-04 Epoch 17/25 120/120 [==============================] - 18s 146ms/step - loss: 0.1731 - accuracy: 0.9342 - val_loss: 0.4373 - val_accuracy: 0.8902 - lr: 6.2500e-05 Epoch 18/25 120/120 [==============================] - ETA: 0s - loss: 0.1698 - accuracy: 0.9355 Epoch 18: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05. 120/120 [==============================] - 17s 140ms/step - loss: 0.1698 - accuracy: 0.9355 - val_loss: 0.4558 - val_accuracy: 0.8715 - lr: 6.2500e-05 Epoch 19/25 120/120 [==============================] - 17s 141ms/step - loss: 0.1718 - accuracy: 0.9389 - val_loss: 0.4159 - val_accuracy: 0.8972 - lr: 3.1250e-05 Epoch 20/25 120/120 [==============================] - 19s 156ms/step - loss: 0.1567 - accuracy: 0.9434 - val_loss: 0.4377 - val_accuracy: 0.8832 - lr: 3.1250e-05 Epoch 21/25 120/120 [==============================] - 19s 156ms/step - loss: 0.1580 - accuracy: 0.9410 - val_loss: 0.4192 - val_accuracy: 0.8995 - lr: 3.1250e-05 Epoch 22/25 120/120 [==============================] - 18s 148ms/step - loss: 0.1533 - accuracy: 0.9405 - val_loss: 0.4188 - val_accuracy: 0.8925 - lr: 3.1250e-05 Epoch 23/25 120/120 [==============================] - 19s 159ms/step - loss: 0.1452 - accuracy: 0.9447 - val_loss: 0.4233 - val_accuracy: 0.9042 - lr: 3.1250e-05 Epoch 24/25 120/120 [==============================] - 19s 158ms/step - loss: 0.1479 - accuracy: 0.9520 - val_loss: 0.4453 - val_accuracy: 0.8738 - lr: 3.1250e-05 Epoch 25/25 120/120 [==============================] - ETA: 0s - loss: 0.1462 - accuracy: 0.9473 Epoch 25: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05. 120/120 [==============================] - 19s 159ms/step - loss: 0.1462 - accuracy: 0.9473 - val_loss: 0.4296 - val_accuracy: 0.8972 - lr: 3.1250e-05
Model Evaluation
plt.plot(lron_da_spe_history.history['accuracy'])
plt.plot(lron_da_spe_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations¶
- We still see signs of the training data overfitting though the validation data is much closer.
- Training and validation are still pretty divergent.
plt.plot(lron_da_spe_history.history['loss'])
plt.plot(lron_da_spe_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations¶
- Loss flattens around epoch 5.
Evaluate the model on test data
lron_da_spe_test_accuracy = lron_da_spe_model.evaluate(X_test_normalized,
y_test_encoded,
#verbose=2
)
15/15 [==============================] - 1s 65ms/step - loss: 0.4318 - accuracy: 0.8968
best_model_accuracy = lron_da_spe_history.history['accuracy'][np.argmin(lron_da_spe_history.history['loss'])]
best_model_accuracy
0.9446920156478882
#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = lron_da_spe_test_accuracy[1]
print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])
accuracy 0.944692 Name: Complex Model with batch normalization, learning rate reduction, basic data augmentation and steps per epoch, dtype: float64 accuracy 0.896842 Name: Complex Model with batch normalization, learning rate reduction, basic data augmentation and steps per epoch, dtype: float64
Observations¶
- Test Accuracy is 90%
- Training model accuracy with least loss is 95%
Plotting the Confusion Matrix
# Complete the code to obtain the output probabilities
y_pred=lron_da_spe_model.predict(X_test_normalized)
15/15 [==============================] - 1s 66ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg) # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Observations¶
- Misclassification of Black-grass has worsened
- Misclassification of Loose Silky-bent has improved
- Most others remain about the same level of misclassification.
Increase the epochs and add early stopping to limit learning¶
In this model we will increase the epochs to try and improve learning, but limit it using the early stopping callback to prevent too many epochs from running.
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
model_name = "Complex Model with batch normalization, learning rate reduction, basic data augmentation, steps per epoch and early stopping"
# Complete the code to set the rotation_range to 20
train_datagen = ImageDataGenerator(
rotation_range=20,
fill_mode='nearest'
)
early_stopping__callback = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=20, min_delta=0.0001, restore_best_weights=True)
h, w, c = X_train_normalized[1000].shape
print('width: ', w)
print('height: ', h)
print('channel:', c)
# Intializing a sequential model
lron_da2_spe_es_model = Sequential()
# create convolution and max-pooling layers with input size of images
lron_da2_spe_es_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h,w,c)))
lron_da2_spe_es_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_da2_spe_es_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
lron_da2_spe_es_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_da2_spe_es_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da2_spe_es_model.add(MaxPooling2D((2, 2), padding = 'same'))
# add BatchNormalization
lron_da2_spe_es_model.add(BatchNormalization())
# create convolution and max-pooling layers
lron_da2_spe_es_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da2_spe_es_model.add(MaxPooling2D((2, 2), padding = 'same'))
# flatten the output of the conv layer
lron_da2_spe_es_model.add(Flatten())
# add a fully connected layer
lron_da2_spe_es_model.add(Dense(128, activation='relu'))
lron_da2_spe_es_model.add(Dropout(0.3))
# add a fully connected layer
lron_da2_spe_es_model.add(Dense(512, activation='relu'))
lron_da2_spe_es_model.add(Dropout(0.3))
# add a fully connected layer
lron_da2_spe_es_model.add(Dense(256, activation='relu'))
lron_da2_spe_es_model.add(Dropout(0.3))
# output layer with 12 neurons and softmax
lron_da2_spe_es_model.add(Dense(12, activation='softmax'))
# Adam Optimizer
opt=Adam()
# Compile the model
lron_da2_spe_es_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# generate the summary
lron_da2_spe_es_model.summary()
width: 64
height: 64
channel: 3
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 32) 896
max_pooling2d (MaxPooling2 (None, 32, 32, 32) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 64) 18496
max_pooling2d_1 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_2 (Conv2D) (None, 16, 16, 128) 73856
max_pooling2d_2 (MaxPoolin (None, 8, 8, 128) 0
g2D)
batch_normalization (Batch (None, 8, 8, 128) 512
Normalization)
conv2d_3 (Conv2D) (None, 8, 8, 128) 147584
max_pooling2d_3 (MaxPoolin (None, 4, 4, 128) 0
g2D)
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 128) 262272
dropout (Dropout) (None, 128) 0
dense_1 (Dense) (None, 512) 66048
dropout_1 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 256) 131328
dropout_2 (Dropout) (None, 256) 0
dense_3 (Dense) (None, 12) 3084
=================================================================
Total params: 704076 (2.69 MB)
Trainable params: 703820 (2.68 MB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Fitting the model on the train data
lron_da2_spe_es_history = lron_da2_spe_es_model.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
shuffle=False),
epochs=100,
steps_per_epoch=X_train_normalized.shape[0] // batch_size, #errors out of range
validation_data=(X_val_normalized,y_val_encoded),
verbose=1,
callbacks=[learning_rate_reduction,early_stopping__callback])
Epoch 1/100 120/120 [==============================] - 14s 105ms/step - loss: 2.0579 - accuracy: 0.2734 - val_loss: 2.4149 - val_accuracy: 0.1893 - lr: 0.0010 Epoch 2/100 120/120 [==============================] - 12s 97ms/step - loss: 1.4825 - accuracy: 0.4747 - val_loss: 2.2907 - val_accuracy: 0.2103 - lr: 0.0010 Epoch 3/100 120/120 [==============================] - 12s 97ms/step - loss: 1.3135 - accuracy: 0.5389 - val_loss: 1.8938 - val_accuracy: 0.5561 - lr: 0.0010 Epoch 4/100 120/120 [==============================] - 12s 98ms/step - loss: 1.1223 - accuracy: 0.6068 - val_loss: 1.5954 - val_accuracy: 0.4089 - lr: 0.0010 Epoch 5/100 120/120 [==============================] - 12s 98ms/step - loss: 0.9872 - accuracy: 0.6569 - val_loss: 1.2421 - val_accuracy: 0.6542 - lr: 0.0010 Epoch 6/100 120/120 [==============================] - 11s 95ms/step - loss: 0.9010 - accuracy: 0.6865 - val_loss: 0.7893 - val_accuracy: 0.7570 - lr: 0.0010 Epoch 7/100 120/120 [==============================] - 11s 94ms/step - loss: 0.8140 - accuracy: 0.7145 - val_loss: 1.7838 - val_accuracy: 0.3902 - lr: 0.0010 Epoch 8/100 120/120 [==============================] - ETA: 0s - loss: 0.7933 - accuracy: 0.7261 Epoch 8: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. 120/120 [==============================] - 11s 92ms/step - loss: 0.7933 - accuracy: 0.7261 - val_loss: 0.7397 - val_accuracy: 0.7523 - lr: 0.0010 Epoch 9/100 120/120 [==============================] - 11s 93ms/step - loss: 0.5773 - accuracy: 0.7940 - val_loss: 0.9857 - val_accuracy: 0.7173 - lr: 5.0000e-04 Epoch 10/100 120/120 [==============================] - ETA: 0s - loss: 0.5624 - accuracy: 0.8042 Epoch 10: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. 120/120 [==============================] - 11s 89ms/step - loss: 0.5624 - accuracy: 0.8042 - val_loss: 0.7674 - val_accuracy: 0.7266 - lr: 5.0000e-04 Epoch 11/100 120/120 [==============================] - 10s 84ms/step - loss: 0.4956 - accuracy: 0.8218 - val_loss: 0.5210 - val_accuracy: 0.8318 - lr: 2.5000e-04 Epoch 12/100 120/120 [==============================] - 11s 93ms/step - loss: 0.4709 - accuracy: 0.8320 - val_loss: 0.5778 - val_accuracy: 0.8178 - lr: 2.5000e-04 Epoch 13/100 120/120 [==============================] - 11s 94ms/step - loss: 0.4226 - accuracy: 0.8503 - val_loss: 0.5153 - val_accuracy: 0.8458 - lr: 2.5000e-04 Epoch 14/100 120/120 [==============================] - 11s 93ms/step - loss: 0.4193 - accuracy: 0.8574 - val_loss: 0.5674 - val_accuracy: 0.8154 - lr: 2.5000e-04 Epoch 15/100 120/120 [==============================] - 11s 94ms/step - loss: 0.4212 - accuracy: 0.8524 - val_loss: 0.5017 - val_accuracy: 0.8668 - lr: 2.5000e-04 Epoch 16/100 120/120 [==============================] - 11s 95ms/step - loss: 0.3924 - accuracy: 0.8595 - val_loss: 0.5353 - val_accuracy: 0.8341 - lr: 2.5000e-04 Epoch 17/100 120/120 [==============================] - ETA: 0s - loss: 0.3769 - accuracy: 0.8624 Epoch 17: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. 120/120 [==============================] - 11s 94ms/step - loss: 0.3769 - accuracy: 0.8624 - val_loss: 0.5655 - val_accuracy: 0.8341 - lr: 2.5000e-04 Epoch 18/100 120/120 [==============================] - 11s 91ms/step - loss: 0.3533 - accuracy: 0.8700 - val_loss: 0.5138 - val_accuracy: 0.8598 - lr: 1.2500e-04 Epoch 19/100 120/120 [==============================] - ETA: 0s - loss: 0.3327 - accuracy: 0.8765 Epoch 19: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05. 120/120 [==============================] - 11s 92ms/step - loss: 0.3327 - accuracy: 0.8765 - val_loss: 0.5110 - val_accuracy: 0.8481 - lr: 1.2500e-04 Epoch 20/100 120/120 [==============================] - 11s 92ms/step - loss: 0.3215 - accuracy: 0.8802 - val_loss: 0.4936 - val_accuracy: 0.8621 - lr: 6.2500e-05 Epoch 21/100 120/120 [==============================] - ETA: 0s - loss: 0.3167 - accuracy: 0.8841 Epoch 21: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05. 120/120 [==============================] - 11s 92ms/step - loss: 0.3167 - accuracy: 0.8841 - val_loss: 0.4775 - val_accuracy: 0.8598 - lr: 6.2500e-05 Epoch 22/100 120/120 [==============================] - 11s 95ms/step - loss: 0.3097 - accuracy: 0.8779 - val_loss: 0.4919 - val_accuracy: 0.8551 - lr: 3.1250e-05 Epoch 23/100 120/120 [==============================] - ETA: 0s - loss: 0.2976 - accuracy: 0.8805 Epoch 23: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05. 120/120 [==============================] - 11s 95ms/step - loss: 0.2976 - accuracy: 0.8805 - val_loss: 0.5168 - val_accuracy: 0.8575 - lr: 3.1250e-05 Epoch 24/100 120/120 [==============================] - 11s 87ms/step - loss: 0.3017 - accuracy: 0.8849 - val_loss: 0.5004 - val_accuracy: 0.8528 - lr: 1.5625e-05 Epoch 25/100 120/120 [==============================] - ETA: 0s - loss: 0.2963 - accuracy: 0.8907 Epoch 25: ReduceLROnPlateau reducing learning rate to 1e-05. 120/120 [==============================] - 11s 88ms/step - loss: 0.2963 - accuracy: 0.8907 - val_loss: 0.4963 - val_accuracy: 0.8481 - lr: 1.5625e-05 Epoch 26/100 120/120 [==============================] - 11s 92ms/step - loss: 0.2998 - accuracy: 0.8904 - val_loss: 0.4889 - val_accuracy: 0.8621 - lr: 1.0000e-05 Epoch 27/100 120/120 [==============================] - 11s 92ms/step - loss: 0.2997 - accuracy: 0.8920 - val_loss: 0.4910 - val_accuracy: 0.8621 - lr: 1.0000e-05 Epoch 28/100 120/120 [==============================] - 11s 91ms/step - loss: 0.3003 - accuracy: 0.8883 - val_loss: 0.4845 - val_accuracy: 0.8621 - lr: 1.0000e-05 Epoch 29/100 120/120 [==============================] - 11s 94ms/step - loss: 0.3036 - accuracy: 0.8891 - val_loss: 0.4937 - val_accuracy: 0.8645 - lr: 1.0000e-05 Epoch 30/100 120/120 [==============================] - 11s 95ms/step - loss: 0.2872 - accuracy: 0.8946 - val_loss: 0.4926 - val_accuracy: 0.8645 - lr: 1.0000e-05 Epoch 31/100 120/120 [==============================] - 11s 94ms/step - loss: 0.2984 - accuracy: 0.8834 - val_loss: 0.4885 - val_accuracy: 0.8621 - lr: 1.0000e-05 Epoch 32/100 120/120 [==============================] - 11s 95ms/step - loss: 0.2692 - accuracy: 0.9009 - val_loss: 0.4908 - val_accuracy: 0.8598 - lr: 1.0000e-05 Epoch 33/100 120/120 [==============================] - 11s 91ms/step - loss: 0.2913 - accuracy: 0.8923 - val_loss: 0.4871 - val_accuracy: 0.8528 - lr: 1.0000e-05 Epoch 34/100 120/120 [==============================] - 11s 90ms/step - loss: 0.2950 - accuracy: 0.8886 - val_loss: 0.4895 - val_accuracy: 0.8645 - lr: 1.0000e-05 Epoch 35/100 120/120 [==============================] - 11s 92ms/step - loss: 0.2759 - accuracy: 0.8949 - val_loss: 0.5009 - val_accuracy: 0.8621 - lr: 1.0000e-05
Model Evaluation
plt.plot(lron_da2_spe_es_history.history['accuracy'])
plt.plot(lron_da2_spe_es_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- From the graph we can see that the training accuracy and validation are reasonably close once epoch 18 is reached.
- this model has the closest trianing and validation we have seen so far.
plt.plot(lron_da2_spe_es_history.history['loss'])
plt.plot(lron_da2_spe_es_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- Loss flattens after epoch 15. I suspect if w elimited this model to 15 epochs we might se very close results.
Evaluate the model on test data
lron_da2_spe_es_accuracy = lron_da2_spe_es_model.evaluate(X_test_normalized,
y_test_encoded,
#verbose=2
)
15/15 [==============================] - 1s 47ms/step - loss: 0.5379 - accuracy: 0.8337
best_model_accuracy = lron_da2_spe_es_history.history['accuracy'][np.argmin(lron_da2_spe_es_history.history['loss'])]
best_model_accuracy
0.9009174108505249
#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = lron_da2_spe_es_accuracy[1]
print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])
accuracy 0.900917 Name: Complex Model with batch normalization, learning rate reduction, basic data augmentation, steps per epoch and early stopping, dtype: float64 accuracy 0.833684 Name: Complex Model with batch normalization, learning rate reduction, basic data augmentation, steps per epoch and early stopping, dtype: float64
Observations¶
- Test Accuracy is 83%
- Training model accuracy with least loss is 90%
Plotting the Confusion Matrix
# Complete the code to obtain the output probabilities
y_pred=lron_da2_spe_es_model.predict(X_test_normalized) #<---- check this is right for others
15/15 [==============================] - 1s 50ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg) # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Observations¶
- Misclassification of Black-grass has worsened.
- Misclassification of Loose Silky-bent has improved.
- Most others have increased level of misclassification.
Add more complex data augmentaion to model¶
In this model we will try using more data augmentaion to improve the model. We will keep everything else the same.
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
train_datagen = ImageDataGenerator(
#rescale=1./255,
shear_range = 0.2,
fill_mode='nearest',
rotation_range=180, # randomly rotate images in the range (was 180)
zoom_range = 0.2, # Randomly zoom image (was.1 with others)
width_shift_range=0.2, # randomly shift images horizontally
height_shift_range=0.2, # randomly shift images vertically
horizontal_flip=True, # randomly flip images horizontally
vertical_flip=True # randomly flip images vertically
)
early_stopping__callback = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=20, min_delta=0.0001, restore_best_weights=True)
model_name = "Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping"
h, w, c = X_train_normalized[1000].shape
print('width: ', w)
print('height: ', h)
print('channel:', c)
# Intializing a sequential model
lron_da2_spe_model = Sequential()
# create convolution and max-pooling layers with input size of images
lron_da2_spe_model.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(h,w,c)))
lron_da2_spe_model.add(BatchNormalization())
lron_da2_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_da2_spe_model.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
lron_da2_spe_model.add(BatchNormalization())
lron_da2_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_da2_spe_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da2_spe_model.add(BatchNormalization())
lron_da2_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))
# create convolution and max-pooling layers
lron_da2_spe_model.add(Conv2D(128, (3, 3), activation='relu', padding="same"))
lron_da2_spe_model.add(BatchNormalization())
lron_da2_spe_model.add(MaxPooling2D((2, 2), padding = 'same'))
# flatten the output of the conv layer
lron_da2_spe_model.add(Flatten())
# add a fully connected layer
lron_da2_spe_model.add(Dense(128, activation='relu'))
lron_da2_spe_model.add(Dropout(0.3))
# add a fully connected layer
lron_da2_spe_model.add(Dense(512, activation='relu'))
lron_da2_spe_model.add(Dropout(0.3))
# add a fully connected layer
lron_da2_spe_model.add(Dense(256, activation='relu'))
lron_da2_spe_model.add(Dropout(0.3))
# output layer with 12 neurons and softmax
lron_da2_spe_model.add(Dense(12, activation='softmax'))
# Adam Optimizer
opt=Adam()
# Compile the model
lron_da2_spe_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# generate the summary
lron_da2_spe_model.summary()
width: 64
height: 64
channel: 3
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 32) 896
batch_normalization (Batch (None, 64, 64, 32) 128
Normalization)
max_pooling2d (MaxPooling2 (None, 32, 32, 32) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 64) 18496
batch_normalization_1 (Bat (None, 32, 32, 64) 256
chNormalization)
max_pooling2d_1 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_2 (Conv2D) (None, 16, 16, 128) 73856
batch_normalization_2 (Bat (None, 16, 16, 128) 512
chNormalization)
max_pooling2d_2 (MaxPoolin (None, 8, 8, 128) 0
g2D)
conv2d_3 (Conv2D) (None, 8, 8, 128) 147584
batch_normalization_3 (Bat (None, 8, 8, 128) 512
chNormalization)
max_pooling2d_3 (MaxPoolin (None, 4, 4, 128) 0
g2D)
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 128) 262272
dropout (Dropout) (None, 128) 0
dense_1 (Dense) (None, 512) 66048
dropout_1 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 256) 131328
dropout_2 (Dropout) (None, 256) 0
dense_3 (Dense) (None, 12) 3084
=================================================================
Total params: 704972 (2.69 MB)
Trainable params: 704268 (2.69 MB)
Non-trainable params: 704 (2.75 KB)
_________________________________________________________________
Fitting the model on the train data
lron_da2_spe_history = lron_da2_spe_model.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
shuffle=False),
epochs=100,
steps_per_epoch=X_train_normalized.shape[0] // batch_size, #errors out of range
validation_data=(X_val_normalized,y_val_encoded),
verbose=1,callbacks=[learning_rate_reduction,early_stopping__callback])
Epoch 1/100 120/120 [==============================] - 18s 141ms/step - loss: 2.1294 - accuracy: 0.2634 - val_loss: 5.2550 - val_accuracy: 0.1379 - lr: 0.0010 Epoch 2/100 120/120 [==============================] - 19s 154ms/step - loss: 1.6919 - accuracy: 0.4204 - val_loss: 6.8600 - val_accuracy: 0.1379 - lr: 0.0010 Epoch 3/100 120/120 [==============================] - 18s 150ms/step - loss: 1.4913 - accuracy: 0.4750 - val_loss: 3.6939 - val_accuracy: 0.1682 - lr: 0.0010 Epoch 4/100 120/120 [==============================] - 18s 152ms/step - loss: 1.3416 - accuracy: 0.5394 - val_loss: 4.3308 - val_accuracy: 0.1449 - lr: 0.0010 Epoch 5/100 120/120 [==============================] - 18s 149ms/step - loss: 1.2554 - accuracy: 0.5740 - val_loss: 1.7323 - val_accuracy: 0.3762 - lr: 0.0010 Epoch 6/100 120/120 [==============================] - 18s 151ms/step - loss: 1.1122 - accuracy: 0.6176 - val_loss: 1.6744 - val_accuracy: 0.4673 - lr: 0.0010 Epoch 7/100 120/120 [==============================] - 18s 146ms/step - loss: 1.0057 - accuracy: 0.6467 - val_loss: 1.4680 - val_accuracy: 0.5584 - lr: 0.0010 Epoch 8/100 120/120 [==============================] - 18s 150ms/step - loss: 0.9692 - accuracy: 0.6731 - val_loss: 1.2056 - val_accuracy: 0.6168 - lr: 0.0010 Epoch 9/100 120/120 [==============================] - 17s 142ms/step - loss: 0.8942 - accuracy: 0.7062 - val_loss: 0.9388 - val_accuracy: 0.6939 - lr: 0.0010 Epoch 10/100 120/120 [==============================] - 19s 154ms/step - loss: 0.8869 - accuracy: 0.7012 - val_loss: 1.2399 - val_accuracy: 0.6098 - lr: 0.0010 Epoch 11/100 120/120 [==============================] - ETA: 0s - loss: 0.7852 - accuracy: 0.7258 Epoch 11: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. 120/120 [==============================] - 18s 148ms/step - loss: 0.7852 - accuracy: 0.7258 - val_loss: 0.9927 - val_accuracy: 0.6846 - lr: 0.0010 Epoch 12/100 120/120 [==============================] - 18s 149ms/step - loss: 0.7010 - accuracy: 0.7554 - val_loss: 0.9238 - val_accuracy: 0.7103 - lr: 5.0000e-04 Epoch 13/100 120/120 [==============================] - 18s 149ms/step - loss: 0.6523 - accuracy: 0.7701 - val_loss: 0.5948 - val_accuracy: 0.8084 - lr: 5.0000e-04 Epoch 14/100 120/120 [==============================] - 19s 158ms/step - loss: 0.6177 - accuracy: 0.7893 - val_loss: 0.5547 - val_accuracy: 0.8294 - lr: 5.0000e-04 Epoch 15/100 120/120 [==============================] - 18s 152ms/step - loss: 0.5863 - accuracy: 0.7937 - val_loss: 0.6298 - val_accuracy: 0.7921 - lr: 5.0000e-04 Epoch 16/100 120/120 [==============================] - 19s 155ms/step - loss: 0.5658 - accuracy: 0.8089 - val_loss: 0.4873 - val_accuracy: 0.8318 - lr: 5.0000e-04 Epoch 17/100 120/120 [==============================] - 17s 140ms/step - loss: 0.5366 - accuracy: 0.8066 - val_loss: 0.7136 - val_accuracy: 0.7593 - lr: 5.0000e-04 Epoch 18/100 120/120 [==============================] - ETA: 0s - loss: 0.5118 - accuracy: 0.8155 Epoch 18: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. 120/120 [==============================] - 17s 141ms/step - loss: 0.5118 - accuracy: 0.8155 - val_loss: 1.4669 - val_accuracy: 0.6332 - lr: 5.0000e-04 Epoch 19/100 120/120 [==============================] - 18s 152ms/step - loss: 0.4726 - accuracy: 0.8393 - val_loss: 0.4200 - val_accuracy: 0.8551 - lr: 2.5000e-04 Epoch 20/100 120/120 [==============================] - 19s 162ms/step - loss: 0.4393 - accuracy: 0.8438 - val_loss: 0.3603 - val_accuracy: 0.8785 - lr: 2.5000e-04 Epoch 21/100 120/120 [==============================] - 18s 151ms/step - loss: 0.4371 - accuracy: 0.8474 - val_loss: 0.3736 - val_accuracy: 0.8762 - lr: 2.5000e-04 Epoch 22/100 120/120 [==============================] - ETA: 0s - loss: 0.4297 - accuracy: 0.8459 Epoch 22: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. 120/120 [==============================] - 18s 150ms/step - loss: 0.4297 - accuracy: 0.8459 - val_loss: 0.6631 - val_accuracy: 0.7874 - lr: 2.5000e-04 Epoch 23/100 120/120 [==============================] - 19s 154ms/step - loss: 0.3933 - accuracy: 0.8535 - val_loss: 0.3623 - val_accuracy: 0.8925 - lr: 1.2500e-04 Epoch 24/100 120/120 [==============================] - 19s 155ms/step - loss: 0.3791 - accuracy: 0.8553 - val_loss: 0.3262 - val_accuracy: 0.9089 - lr: 1.2500e-04 Epoch 25/100 120/120 [==============================] - 19s 154ms/step - loss: 0.3734 - accuracy: 0.8676 - val_loss: 0.3479 - val_accuracy: 0.8925 - lr: 1.2500e-04 Epoch 26/100 120/120 [==============================] - ETA: 0s - loss: 0.3758 - accuracy: 0.8621 Epoch 26: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05. 120/120 [==============================] - 18s 152ms/step - loss: 0.3758 - accuracy: 0.8621 - val_loss: 0.4046 - val_accuracy: 0.8575 - lr: 1.2500e-04 Epoch 27/100 120/120 [==============================] - 18s 151ms/step - loss: 0.3534 - accuracy: 0.8747 - val_loss: 0.3166 - val_accuracy: 0.9089 - lr: 6.2500e-05 Epoch 28/100 120/120 [==============================] - 16s 136ms/step - loss: 0.3565 - accuracy: 0.8710 - val_loss: 0.3039 - val_accuracy: 0.9206 - lr: 6.2500e-05 Epoch 29/100 120/120 [==============================] - 18s 149ms/step - loss: 0.3589 - accuracy: 0.8679 - val_loss: 0.3348 - val_accuracy: 0.8925 - lr: 6.2500e-05 Epoch 30/100 120/120 [==============================] - ETA: 0s - loss: 0.3312 - accuracy: 0.8820 Epoch 30: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05. 120/120 [==============================] - 19s 155ms/step - loss: 0.3312 - accuracy: 0.8820 - val_loss: 0.3366 - val_accuracy: 0.9019 - lr: 6.2500e-05 Epoch 31/100 120/120 [==============================] - 19s 158ms/step - loss: 0.3349 - accuracy: 0.8765 - val_loss: 0.3172 - val_accuracy: 0.9089 - lr: 3.1250e-05 Epoch 32/100 120/120 [==============================] - ETA: 0s - loss: 0.3245 - accuracy: 0.8860 Epoch 32: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05. 120/120 [==============================] - 18s 150ms/step - loss: 0.3245 - accuracy: 0.8860 - val_loss: 0.3039 - val_accuracy: 0.9019 - lr: 3.1250e-05 Epoch 33/100 120/120 [==============================] - 19s 161ms/step - loss: 0.3343 - accuracy: 0.8826 - val_loss: 0.2923 - val_accuracy: 0.9159 - lr: 1.5625e-05 Epoch 34/100 120/120 [==============================] - 19s 160ms/step - loss: 0.3300 - accuracy: 0.8744 - val_loss: 0.2968 - val_accuracy: 0.9252 - lr: 1.5625e-05 Epoch 35/100 120/120 [==============================] - 18s 152ms/step - loss: 0.3088 - accuracy: 0.8878 - val_loss: 0.2976 - val_accuracy: 0.9112 - lr: 1.5625e-05 Epoch 36/100 120/120 [==============================] - ETA: 0s - loss: 0.3067 - accuracy: 0.8868 Epoch 36: ReduceLROnPlateau reducing learning rate to 1e-05. 120/120 [==============================] - 18s 151ms/step - loss: 0.3067 - accuracy: 0.8868 - val_loss: 0.3018 - val_accuracy: 0.9065 - lr: 1.5625e-05 Epoch 37/100 120/120 [==============================] - 19s 154ms/step - loss: 0.3057 - accuracy: 0.8873 - val_loss: 0.3005 - val_accuracy: 0.9089 - lr: 1.0000e-05 Epoch 38/100 120/120 [==============================] - 19s 156ms/step - loss: 0.3283 - accuracy: 0.8823 - val_loss: 0.3004 - val_accuracy: 0.9206 - lr: 1.0000e-05 Epoch 39/100 120/120 [==============================] - 18s 149ms/step - loss: 0.3083 - accuracy: 0.8915 - val_loss: 0.2974 - val_accuracy: 0.9182 - lr: 1.0000e-05 Epoch 40/100 120/120 [==============================] - 17s 143ms/step - loss: 0.3073 - accuracy: 0.8855 - val_loss: 0.3021 - val_accuracy: 0.9089 - lr: 1.0000e-05 Epoch 41/100 120/120 [==============================] - 16s 132ms/step - loss: 0.3086 - accuracy: 0.8844 - val_loss: 0.3040 - val_accuracy: 0.8995 - lr: 1.0000e-05 Epoch 42/100 120/120 [==============================] - 16s 133ms/step - loss: 0.3187 - accuracy: 0.8789 - val_loss: 0.3080 - val_accuracy: 0.8972 - lr: 1.0000e-05 Epoch 43/100 120/120 [==============================] - 17s 143ms/step - loss: 0.3060 - accuracy: 0.8931 - val_loss: 0.3092 - val_accuracy: 0.8995 - lr: 1.0000e-05 Epoch 44/100 120/120 [==============================] - 18s 149ms/step - loss: 0.3206 - accuracy: 0.8760 - val_loss: 0.3026 - val_accuracy: 0.9206 - lr: 1.0000e-05 Epoch 45/100 120/120 [==============================] - 18s 153ms/step - loss: 0.3113 - accuracy: 0.8865 - val_loss: 0.3064 - val_accuracy: 0.9112 - lr: 1.0000e-05 Epoch 46/100 120/120 [==============================] - 19s 158ms/step - loss: 0.3138 - accuracy: 0.8875 - val_loss: 0.3136 - val_accuracy: 0.8972 - lr: 1.0000e-05 Epoch 47/100 120/120 [==============================] - 19s 157ms/step - loss: 0.3208 - accuracy: 0.8799 - val_loss: 0.3089 - val_accuracy: 0.8995 - lr: 1.0000e-05 Epoch 48/100 120/120 [==============================] - 18s 152ms/step - loss: 0.3110 - accuracy: 0.8855 - val_loss: 0.3066 - val_accuracy: 0.9112 - lr: 1.0000e-05 Epoch 49/100 120/120 [==============================] - 19s 157ms/step - loss: 0.3132 - accuracy: 0.8931 - val_loss: 0.3047 - val_accuracy: 0.9112 - lr: 1.0000e-05 Epoch 50/100 120/120 [==============================] - 18s 150ms/step - loss: 0.3009 - accuracy: 0.8865 - val_loss: 0.3015 - val_accuracy: 0.9136 - lr: 1.0000e-05 Epoch 51/100 120/120 [==============================] - 19s 154ms/step - loss: 0.3149 - accuracy: 0.8899 - val_loss: 0.3044 - val_accuracy: 0.9065 - lr: 1.0000e-05 Epoch 52/100 120/120 [==============================] - 19s 158ms/step - loss: 0.3048 - accuracy: 0.8896 - val_loss: 0.3044 - val_accuracy: 0.9065 - lr: 1.0000e-05 Epoch 53/100 120/120 [==============================] - 18s 154ms/step - loss: 0.3092 - accuracy: 0.8883 - val_loss: 0.3014 - val_accuracy: 0.9089 - lr: 1.0000e-05 Epoch 54/100 120/120 [==============================] - 18s 150ms/step - loss: 0.3041 - accuracy: 0.8839 - val_loss: 0.3031 - val_accuracy: 0.9089 - lr: 1.0000e-05
Model Evaluation
plt.plot(lron_da2_spe_history.history['accuracy'])
plt.plot(lron_da2_spe_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- From the graph we can see that the training and the validation accuracy was very close.
- Validation accuracy was higher than training in this model.
plt.plot(lron_da2_spe_history.history['loss'])
plt.plot(lron_da2_spe_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- Loss for training and accuracy were lockstep after epoch 22.
Evaluate the model on test data
lron_da2_spe_test_accuracy = lron_da2_spe_model.evaluate(X_test_normalized,
y_test_encoded,
#verbose=2
)
15/15 [==============================] - 1s 62ms/step - loss: 0.3626 - accuracy: 0.8758
best_model_accuracy = lron_da2_spe_history.history['accuracy'][np.argmin(lron_da2_spe_history.history['loss'])]
best_model_accuracy
0.8865006566047668
#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = lron_da2_spe_test_accuracy[1]
print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])
accuracy 0.886501 Name: Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping, dtype: float64 accuracy 0.875789 Name: Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping, dtype: float64
Observations¶
- Test Accuracy is 89%
- Training model accuracy with least loss is 88%
- This is our closest model thus far, but we do have models that have better accuracy.
Plotting the Confusion Matrix
# Complete the code to obtain the output probabilities
y_pred=lron_da2_spe_model.predict(X_test_normalized)
15/15 [==============================] - 1s 61ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg) # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Observations¶
- Misclassification of Black-grass and Loose Silky-bent has worsened
- Most others have varied improvement/worsening of misclassification.
- This model is still not suitable.
Model using VGG16 pre-trained model¶
In this model we will use transfer learning from the VGG16 model to try and improve our classification. We will only take the bottom layers and of course make VGG16 non-trainable. We will add our own dense layers.
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
model_name = "VGG16 Model"
early_stopping__callback = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=20, min_delta=0.0001, restore_best_weights=True)
from tensorflow.keras.models import Model
from keras.applications.vgg16 import VGG16
h, w, c = X_train_normalized[1000].shape
print('width: ', w)
print('height: ', h)
print('channel:', c)
vgg_model = VGG16(weights='imagenet', include_top = False, input_shape = (h,w,c))
vgg_model.summary()
width: 64
height: 64
channel: 3
Model: "vgg16"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 64, 64, 3)] 0
block1_conv1 (Conv2D) (None, 64, 64, 64) 1792
block1_conv2 (Conv2D) (None, 64, 64, 64) 36928
block1_pool (MaxPooling2D) (None, 32, 32, 64) 0
block2_conv1 (Conv2D) (None, 32, 32, 128) 73856
block2_conv2 (Conv2D) (None, 32, 32, 128) 147584
block2_pool (MaxPooling2D) (None, 16, 16, 128) 0
block3_conv1 (Conv2D) (None, 16, 16, 256) 295168
block3_conv2 (Conv2D) (None, 16, 16, 256) 590080
block3_conv3 (Conv2D) (None, 16, 16, 256) 590080
block3_pool (MaxPooling2D) (None, 8, 8, 256) 0
block4_conv1 (Conv2D) (None, 8, 8, 512) 1180160
block4_conv2 (Conv2D) (None, 8, 8, 512) 2359808
block4_conv3 (Conv2D) (None, 8, 8, 512) 2359808
block4_pool (MaxPooling2D) (None, 4, 4, 512) 0
block5_conv1 (Conv2D) (None, 4, 4, 512) 2359808
block5_conv2 (Conv2D) (None, 4, 4, 512) 2359808
block5_conv3 (Conv2D) (None, 4, 4, 512) 2359808
block5_pool (MaxPooling2D) (None, 2, 2, 512) 0
=================================================================
Total params: 14714688 (56.13 MB)
Trainable params: 14714688 (56.13 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
# Making all the layers of the VGG model non-trainable. i.e. freezing them
for layer in vgg_model.layers:
layer.trainable = False
vgg_new_model = Sequential()
# Adding the convolutional part of the VGG16 model from above
vgg_new_model.add(vgg_model)
# Flattening the output of the VGG16 model because it is from a convolutional layer
vgg_new_model.add(Flatten())
# Adding a fully connected dense layer with 16 neurons
vgg_new_model.add(Dense(128, activation='relu'))
vgg_new_model.add(Dropout(0.3))
# Adding a fully connected dense layer with 16 neurons
vgg_new_model.add(Dense(512, activation='relu'))
vgg_new_model.add(Dropout(0.3))
vgg_new_model.add(Dense(256, activation='relu'))
vgg_new_model.add(Dropout(0.3))
# output layer with 12 neurons
vgg_new_model.add(Dense(12, activation='softmax'))
# initialize Adam Optimimzer
opt=Adam()
adam_opt = optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)
# Compile model
vgg_new_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
vgg_new_model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
vgg16 (Functional) (None, 2, 2, 512) 14714688
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 128) 262272
dropout (Dropout) (None, 128) 0
dense_1 (Dense) (None, 512) 66048
dropout_1 (Dropout) (None, 512) 0
dense_2 (Dense) (None, 256) 131328
dropout_2 (Dropout) (None, 256) 0
dense_3 (Dense) (None, 12) 3084
=================================================================
Total params: 15177420 (57.90 MB)
Trainable params: 462732 (1.77 MB)
Non-trainable params: 14714688 (56.13 MB)
_________________________________________________________________
history_vgg16 = vgg_new_model.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=32,
seed=42,
shuffle=False),
epochs=100,
steps_per_epoch=math.floor(X_train_normalized.shape[0] / batch_size), #error again
validation_data=(X_val_normalized,y_val_encoded),
verbose=1,callbacks=[learning_rate_reduction,early_stopping__callback])
Epoch 1/100 120/120 [==============================] - 24s 195ms/step - loss: 2.2127 - accuracy: 0.2173 - val_loss: 1.9076 - val_accuracy: 0.3294 - lr: 0.0010 Epoch 2/100 120/120 [==============================] - 23s 190ms/step - loss: 1.8482 - accuracy: 0.3394 - val_loss: 1.7546 - val_accuracy: 0.3785 - lr: 0.0010 Epoch 3/100 120/120 [==============================] - 23s 191ms/step - loss: 1.7462 - accuracy: 0.3641 - val_loss: 1.6430 - val_accuracy: 0.3925 - lr: 0.0010 Epoch 4/100 120/120 [==============================] - 23s 191ms/step - loss: 1.6884 - accuracy: 0.3872 - val_loss: 1.5719 - val_accuracy: 0.4299 - lr: 0.0010 Epoch 5/100 120/120 [==============================] - 22s 186ms/step - loss: 1.6371 - accuracy: 0.4128 - val_loss: 1.5323 - val_accuracy: 0.4673 - lr: 0.0010 Epoch 6/100 120/120 [==============================] - 23s 196ms/step - loss: 1.6326 - accuracy: 0.4121 - val_loss: 1.5059 - val_accuracy: 0.4720 - lr: 0.0010 Epoch 7/100 120/120 [==============================] - 23s 192ms/step - loss: 1.5733 - accuracy: 0.4406 - val_loss: 1.5034 - val_accuracy: 0.4346 - lr: 0.0010 Epoch 8/100 120/120 [==============================] - ETA: 0s - loss: 1.5758 - accuracy: 0.4333 Epoch 8: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. 120/120 [==============================] - 24s 203ms/step - loss: 1.5758 - accuracy: 0.4333 - val_loss: 1.4820 - val_accuracy: 0.4579 - lr: 0.0010 Epoch 9/100 120/120 [==============================] - 24s 202ms/step - loss: 1.4924 - accuracy: 0.4658 - val_loss: 1.4744 - val_accuracy: 0.4696 - lr: 5.0000e-04 Epoch 10/100 120/120 [==============================] - 24s 202ms/step - loss: 1.4700 - accuracy: 0.4820 - val_loss: 1.3961 - val_accuracy: 0.4953 - lr: 5.0000e-04 Epoch 11/100 120/120 [==============================] - 24s 196ms/step - loss: 1.4724 - accuracy: 0.4755 - val_loss: 1.3956 - val_accuracy: 0.5117 - lr: 5.0000e-04 Epoch 12/100 120/120 [==============================] - 23s 193ms/step - loss: 1.4667 - accuracy: 0.4713 - val_loss: 1.3596 - val_accuracy: 0.5187 - lr: 5.0000e-04 Epoch 13/100 120/120 [==============================] - 24s 199ms/step - loss: 1.4327 - accuracy: 0.4799 - val_loss: 1.3718 - val_accuracy: 0.4836 - lr: 5.0000e-04 Epoch 14/100 120/120 [==============================] - ETA: 0s - loss: 1.4235 - accuracy: 0.5001 Epoch 14: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. 120/120 [==============================] - 23s 191ms/step - loss: 1.4235 - accuracy: 0.5001 - val_loss: 1.3431 - val_accuracy: 0.5047 - lr: 5.0000e-04 Epoch 15/100 120/120 [==============================] - 24s 197ms/step - loss: 1.3934 - accuracy: 0.5054 - val_loss: 1.2918 - val_accuracy: 0.5327 - lr: 2.5000e-04 Epoch 16/100 120/120 [==============================] - 24s 200ms/step - loss: 1.3718 - accuracy: 0.5122 - val_loss: 1.3285 - val_accuracy: 0.5164 - lr: 2.5000e-04 Epoch 17/100 120/120 [==============================] - ETA: 0s - loss: 1.3753 - accuracy: 0.5132 Epoch 17: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. 120/120 [==============================] - 23s 193ms/step - loss: 1.3753 - accuracy: 0.5132 - val_loss: 1.3135 - val_accuracy: 0.5304 - lr: 2.5000e-04 Epoch 18/100 120/120 [==============================] - 23s 194ms/step - loss: 1.3599 - accuracy: 0.5098 - val_loss: 1.2818 - val_accuracy: 0.5467 - lr: 1.2500e-04 Epoch 19/100 120/120 [==============================] - 23s 191ms/step - loss: 1.3330 - accuracy: 0.5203 - val_loss: 1.2627 - val_accuracy: 0.5537 - lr: 1.2500e-04 Epoch 20/100 120/120 [==============================] - 24s 203ms/step - loss: 1.3244 - accuracy: 0.5274 - val_loss: 1.2748 - val_accuracy: 0.5421 - lr: 1.2500e-04 Epoch 21/100 120/120 [==============================] - 24s 197ms/step - loss: 1.3323 - accuracy: 0.5250 - val_loss: 1.2601 - val_accuracy: 0.5654 - lr: 1.2500e-04 Epoch 22/100 120/120 [==============================] - 23s 193ms/step - loss: 1.3297 - accuracy: 0.5177 - val_loss: 1.2642 - val_accuracy: 0.5444 - lr: 1.2500e-04 Epoch 23/100 120/120 [==============================] - ETA: 0s - loss: 1.3366 - accuracy: 0.5190 Epoch 23: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05. 120/120 [==============================] - 25s 208ms/step - loss: 1.3366 - accuracy: 0.5190 - val_loss: 1.2741 - val_accuracy: 0.5327 - lr: 1.2500e-04 Epoch 24/100 120/120 [==============================] - 25s 210ms/step - loss: 1.3334 - accuracy: 0.5208 - val_loss: 1.2652 - val_accuracy: 0.5444 - lr: 6.2500e-05 Epoch 25/100 120/120 [==============================] - ETA: 0s - loss: 1.3295 - accuracy: 0.5245 Epoch 25: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05. 120/120 [==============================] - 26s 212ms/step - loss: 1.3295 - accuracy: 0.5245 - val_loss: 1.2524 - val_accuracy: 0.5561 - lr: 6.2500e-05 Epoch 26/100 120/120 [==============================] - 24s 200ms/step - loss: 1.3380 - accuracy: 0.5169 - val_loss: 1.2521 - val_accuracy: 0.5537 - lr: 3.1250e-05 Epoch 27/100 120/120 [==============================] - ETA: 0s - loss: 1.3091 - accuracy: 0.5376 Epoch 27: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05. 120/120 [==============================] - 23s 194ms/step - loss: 1.3091 - accuracy: 0.5376 - val_loss: 1.2456 - val_accuracy: 0.5537 - lr: 3.1250e-05 Epoch 28/100 120/120 [==============================] - 24s 201ms/step - loss: 1.2979 - accuracy: 0.5368 - val_loss: 1.2439 - val_accuracy: 0.5584 - lr: 1.5625e-05 Epoch 29/100 120/120 [==============================] - ETA: 0s - loss: 1.3247 - accuracy: 0.5198 Epoch 29: ReduceLROnPlateau reducing learning rate to 1e-05. 120/120 [==============================] - 24s 200ms/step - loss: 1.3247 - accuracy: 0.5198 - val_loss: 1.2445 - val_accuracy: 0.5561 - lr: 1.5625e-05 Epoch 30/100 120/120 [==============================] - 25s 207ms/step - loss: 1.3144 - accuracy: 0.5263 - val_loss: 1.2426 - val_accuracy: 0.5561 - lr: 1.0000e-05 Epoch 31/100 120/120 [==============================] - 24s 200ms/step - loss: 1.3043 - accuracy: 0.5400 - val_loss: 1.2438 - val_accuracy: 0.5491 - lr: 1.0000e-05 Epoch 32/100 120/120 [==============================] - 24s 201ms/step - loss: 1.3245 - accuracy: 0.5174 - val_loss: 1.2439 - val_accuracy: 0.5584 - lr: 1.0000e-05 Epoch 33/100 120/120 [==============================] - 24s 200ms/step - loss: 1.3060 - accuracy: 0.5279 - val_loss: 1.2453 - val_accuracy: 0.5561 - lr: 1.0000e-05 Epoch 34/100 120/120 [==============================] - 23s 194ms/step - loss: 1.3125 - accuracy: 0.5347 - val_loss: 1.2475 - val_accuracy: 0.5537 - lr: 1.0000e-05 Epoch 35/100 120/120 [==============================] - 23s 194ms/step - loss: 1.3189 - accuracy: 0.5261 - val_loss: 1.2461 - val_accuracy: 0.5537 - lr: 1.0000e-05 Epoch 36/100 120/120 [==============================] - 24s 199ms/step - loss: 1.3034 - accuracy: 0.5308 - val_loss: 1.2456 - val_accuracy: 0.5561 - lr: 1.0000e-05 Epoch 37/100 120/120 [==============================] - 25s 212ms/step - loss: 1.3280 - accuracy: 0.5316 - val_loss: 1.2453 - val_accuracy: 0.5561 - lr: 1.0000e-05 Epoch 38/100 120/120 [==============================] - 24s 196ms/step - loss: 1.3096 - accuracy: 0.5326 - val_loss: 1.2452 - val_accuracy: 0.5561 - lr: 1.0000e-05 Epoch 39/100 120/120 [==============================] - 25s 206ms/step - loss: 1.2940 - accuracy: 0.5457 - val_loss: 1.2454 - val_accuracy: 0.5537 - lr: 1.0000e-05 Epoch 40/100 120/120 [==============================] - 25s 208ms/step - loss: 1.2821 - accuracy: 0.5371 - val_loss: 1.2417 - val_accuracy: 0.5561 - lr: 1.0000e-05 Epoch 41/100 120/120 [==============================] - 25s 210ms/step - loss: 1.3148 - accuracy: 0.5263 - val_loss: 1.2415 - val_accuracy: 0.5467 - lr: 1.0000e-05
Model Evaluation
plt.plot(history_vgg16.history['accuracy'])
plt.plot(history_vgg16.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:¶
- This model never achieves even 60% accuracy, and thus is a poor model compared to the other ones we have.
plt.plot(history_vgg16.history['loss'])
plt.plot(history_vgg16.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Evaluate the model on test data
vgg_accuracy = vgg_new_model.evaluate(X_test_normalized,
y_test_encoded,
#verbose=2
)
15/15 [==============================] - 3s 175ms/step - loss: 1.2491 - accuracy: 0.5642
best_model_accuracy = history_vgg16.history['accuracy'][np.argmin(history_vgg16.history['loss'])]
best_model_accuracy
0.5370904207229614
#Save metrics for later comparison
training_metrics.loc[model_name] = best_model_accuracy
test_metrics.loc[model_name] = vgg_accuracy[1]
print(training_metrics.loc[model_name])
print(test_metrics.loc[model_name])
accuracy 0.53709 Name: VGG16 Model, dtype: float64 accuracy 0.564211 Name: VGG16 Model, dtype: float64
Observations¶
- Test Accuracy is 56%
- Training model accuracy with least loss is 54%
- This model performed poorly, it may be too generalized
Plotting the Confusion Matrix
# Complete the code to obtain the output probabilities
y_pred=vgg_new_model.predict(X_test_normalized)
15/15 [==============================] - 3s 173ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg) # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Observations¶
- This model did not perform well at all, Misclassification of many plants that were even bettwr identified with our basic model. This model may be too generalized or the dense layers are poor.
training_metrics
| accuracy | |
|---|---|
| Base Model | 0.700026 |
| Complex Model | 0.971146 |
| Complex Model with Class Weights applied | 0.939173 |
| Complex Model with Adam optimizer adjusted | 0.949051 |
| Complex Model with learning rate reduction and batch normalization | 0.985703 |
| Complex Model with batch normalization, learning rate reduction and basic data augmentation | 0.966467 |
| Complex Model with batch normalization, learning rate reduction, basic data augmentation and steps per epoch | 0.944692 |
| Complex Model with batch normalization, learning rate reduction, basic data augmentation, steps per epoch and early stopping | 0.900917 |
| Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping | 0.886501 |
| VGG16 Model | 0.537090 |
test_metrics
| accuracy | |
|---|---|
| Base Model | 0.732632 |
| Complex Model | 0.821053 |
| Complex Model with Class Weights applied | 0.795789 |
| Complex Model with Adam optimizer adjusted | 0.840000 |
| Complex Model with learning rate reduction and batch normalization | 0.911579 |
| Complex Model with batch normalization, learning rate reduction and basic data augmentation | 0.894737 |
| Complex Model with batch normalization, learning rate reduction, basic data augmentation and steps per epoch | 0.896842 |
| Complex Model with batch normalization, learning rate reduction, basic data augmentation, steps per epoch and early stopping | 0.833684 |
| Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping | 0.875789 |
| VGG16 Model | 0.564211 |
Final Model¶
While the "Complex Model with batch normalization, learning rate reduction, complex data augmentation, steps per epoch and early stopping" was very close on the training and test scorres, for this exercise we want the highest accuracy. that came from model "Complex Model with learning rate reduction and batch normalization" with a 91% accuracy on test data. This will be our final model.
lron_base_model_test_accuracy = lron_base_model.evaluate(X_test_normalized,
y_test_encoded,
#verbose=2
)
15/15 [==============================] - 1s 71ms/step - loss: 0.4490 - accuracy: 0.9116
best_model_accuracy = lron_base_history.history['accuracy'][np.argmin(lron_base_history.history['loss'])]
best_model_accuracy
0.98570317029953
Observations¶
- Test Accuracy is 91%
- Training model accuracy with least loss is 99%.
Plotting the Confusion Matrix
# Complete the code to obtain the output probabilities
y_pred=lron_base_model.predict(X_test_normalized)
15/15 [==============================] - 1s 71ms/step
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg) # Complete the code to obatin the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
Visualizing the prediction¶
#lron_da_model until I have another model
h, w, c = X_train_normalized[1000].shape
print('width: ', w)
print('height: ', h)
print('channel:', c)
# Visualizing the predicted and correct label of images from test data
plt.figure(figsize=(2,2))
plt.imshow(X_test[2])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(lron_base_model.predict((X_test_normalized[2].reshape(1,h,w,c))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[2]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[33])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(lron_base_model.predict((X_test_normalized[33].reshape(1,h,w,c))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[33]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[59],)
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(lron_base_model.predict((X_test_normalized[59].reshape(1,h,w,c))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[59]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[36])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(lron_base_model.predict((X_test_normalized[36].reshape(1,h,w,c))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[36]) # using inverse_transform() to get the output label from the output vector
width: 64 height: 64 channel: 3
1/1 [==============================] - 0s 37ms/step Predicted Label ['Small-flowered Cranesbill'] True Label Small-flowered Cranesbill
1/1 [==============================] - 0s 34ms/step Predicted Label ['Cleavers'] True Label Cleavers
1/1 [==============================] - 0s 33ms/step Predicted Label ['Common Chickweed'] True Label Common Chickweed
1/1 [==============================] - 0s 24ms/step Predicted Label ['Shepherds Purse'] True Label Shepherds Purse
Actionable Insights and Business Recommendations¶
# Plotting the classification report
cr=classification_report(y_test_arg,y_pred_arg,target_names=labels_plants)
print(cr)
precision recall f1-score support
Black-grass 0.70 0.54 0.61 26
Charlock 0.97 0.90 0.93 39
Cleavers 0.84 0.93 0.89 29
Common Chickweed 0.97 0.97 0.97 61
Common wheat 0.95 0.95 0.95 22
Fat Hen 0.98 0.98 0.98 48
Loose Silky-bent 0.80 0.91 0.85 65
Maize 0.88 1.00 0.94 22
Scentless Mayweed 0.93 0.96 0.94 52
Shepherds Purse 0.90 0.78 0.84 23
Small-flowered Cranesbill 0.96 0.92 0.94 50
Sugar beet 1.00 0.92 0.96 38
accuracy 0.91 475
macro avg 0.91 0.90 0.90 475
weighted avg 0.91 0.91 0.91 475
Insights:¶
This model does a good job of classifying these plants, for the most part. Several of the models had difficulty with Black-grass and Loose Silky-bent, but this model seems to mostly have difficulty with the Black-grass. Precision is above 90% for many plants, meaning that for those plants we are very accurate.
Having more samples of the lower represented plant classes would likely help with improving classification, as would further epochs , though from experimentation, not by a lot. In other models not included here, it was seen that not resizing images also helps with identification, however, this takes a lot longer to train the models so the loss we get by reducing image size may offset that.
Overall, a 91% accuracy rating feels like a good result, and a suitable model for use.